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Executive  Summary 


This  research  task  (RT-168)  is  addressing  research  needs  defined  by  the  United  States  (US)  Army 
Research,  Development  and  Engineering  Command  (RDECOM)  Armament  Research, 
Development  and  Engineering  Center  (ARDEC)  in  Picatinny,  NJ.  The  purpose  of  this  RT-168 
Phase  I  final  technical  report  is  to  document  the  refinement  and  expansion  of  those  needs  and 
the  status  of  working  sessions,  demonstrations,  presentations,  and  reports  provided  to  the 
ARDEC  team.  These  needs  are  characterized  as  overarching  objectives  and  goals  to  elicit 
requirements  for  the  Armament  Virtual  Collaboratory  Environment  (AVCE)  integrated  Model 
Based  Environment  (iMBE).  The  AVCE  iMBE  is  ARDEC's  envisioned  concept  of  an  integrated 
modeling  environment  -  "the  system  for  designing  future  ARDEC  systems  or  systems-of- 
systems."  The  intent  is  to  understand  the  relationships  between  Systems  Engineering  (SE) 
activities  and  methods  in  the  context  of  a  Digital  Thread  concept  developed  by  ARDEC. 

This  research  tasks  focus  on  the  ARDEC-relevant  needs  for  a  transformation  for  systems 
engineering  enabled  by  model-centric  engineering  (MCE).  Model-centric  engineering1  can  be 
characterized  as  an  overarching  digital  engineering  approach  that  integrates  different  model 
types  with  simulations,  surrogates,  systems  and  components  at  different  levels  of  abstraction 
and  fidelity  across  disciplines  throughout  the  lifecycle.  Industry  is  trending  towards  more 
integration  of  computational  capabilities,  models,  software,  hardware,  platforms,  and  humans- 
in-the-loop.  The  integrated  perspectives  provide  cross-domain  views  for  rapid  system  level 
analysis  allowing  engineers  from  various  disciplines  using  dynamic  models  and  surrogates  to 
support  continuous  and  often  virtual  verification  and  validation  for  tradespace  decisions  in  the 
face  of  changing  mission  needs. 

The  path  forward  has  challenges  but  also  many  opportunities,  both  technical  and 
sociotechnical.  It  must  include  a  modeling  framework  and  consider  the  use  of  high  performance 
computing  (HPC)  that  enables  single  source  of  truth  (SST),  integration  and  interoperability  of 
multi-domain  and  multi-physics  models,  and  provide  for  methods  for  model  integrity  (trust  in 
the  modeling  and  simulating  predictions).  The  modeling  and  infrastructure  for  AVCE  iMBE  is  a 
critical  step  to  enable  a  SST.  While  there  are  literally  thousands  of  tools,  with  about  100  at 
ARDEC,  they  are  often  federated  and  there  is  no  one  single  solution  that  is  fully  integrated  that 
can  be  purchased.  Every  organization  often  has  to  architect  and  engineer  their  model-centric 
engineering  environment.  Most,  like  ARDEC  have  selected  commercial  tools  that  must  be 
integrated  with  many  specialized  tools  that  they  have  developed  for  ARDEC-specific  needs. 

In  order  to  better  understand  the  requirements  for  the  AVCE  iMBE,  ARDEC  initially  had  three 
challenge  areas,  which  has  been  extended  to  five  challenge  areas.  The  SERC  research  team  is 
involved  in  four  of  the  five  challenge  areas.  A  theme  for  a  case  study  involves  Unmanned  Aerial 
Systems  in  which  to  investigate  the  following  five  tasks: 


1  DASD  has  increased  the  emphasis  on  using  the  term  Digital  Engineering.  A  draft  definition  provided  by  the 
Defense  Acquisition  University  (DAU)  for  DE  is:  An  integrated  digital  approach  that  uses  authoritative  sources  of 
systems'  data  and  models  as  a  continuum  across  disciplines  to  support  lifecycle  activities  from  concept  through 
disposal.  This  definition  is  similar  to  working  definition  used  throughout  our  prior  research  task  RT- 
48/118/141/157/170  for  Model  Centric  Engineering  (MCE). 
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■  Task  1:  Framework/architecture  of  development  and  collaboration  environment  that 
support  cross-domain  integration  of  models  to  address  the  heterogeneity  of  the  various 
tools  and  environments 

■  Task  2:  Formalization  of  an  information  model  for  ARDEC-relevant  domains  to  support 
capturing  and  sharing  of  data 

■  Task  3:  Technology  and  domain-relevant  modeling  methodologies 

■  Task  4:  Demonstrations  in  the  context  of  ARDEC-relevant  Challenge  Areas  relevant  to 
Tasks  1,  2,  3  &  5 

■  Task  5:  System  Engineering  Transformation  Roadmap  to  roll  out  capabilities  addressing 
all  five  perspectives  in  parallel: 

o  Technologies  and  infrastructure 
o  Methodologies  and  processes 

o  People,  training,  competencies  and  framework  viewpoints  and  interfaces 
o  Operational  &  contractual  paradigms  for  transformed  interactions  with  industry 
o  Governance 

These  five  tasks  have  been  mapped  to  a  set  of  research  uses  cases,  which  are  detailed  in 
Section  2  of  this  report.  Part  II  of  this  report,  Sections  3  through  14  provide  details  on  each  of 
the  research  use  cases.  The  specific  accomplishments  include,  but  are  not  limited  to  informing 
our  ARDEC  sponsors  through  five  working  sessions,  one  special  session  and  19  virtual  meetings, 
where  we  have  conducted  presentation  and  demonstrations  on  many  topics  such  as:  Model 
Centric  Engineering,  modeling  methodologies,  Model  Frameworks  and  Verification  Tools  for 
Cyber  Physical  Systems  design,  Multidisciplinary  Design  Analysis  and  Optimization,  Decision 
Framework  Approach  and  High  Level  Architecture  (HLA)  for  Virtual  Reality  (VR)  Forces 
demonstrations,  mission  and  system  simulations  with  upstream/downstream  data  interfaces 
demonstrations,  and  graphical  CONOPS  simulations  with  gaming  technology.  One  of  the  high 
potential  areas  involves  research  in  semantic  web  technologies  and  ontologies  as  a  promising 
approach  to  enable  cross-domain  model  through  interoperability  supporting  the  capability  to 
enable  a  single  source  of  truth. 

Finally,  this  research  is  being  conducted  in  collaboration  with  two  SERC  research  tasks 
sponsored  by  the  Naval  Air  Systems  Command  (NAVAIR)  under  RT-170  and  RT-176,  as  well  as 
Department  of  Defense  (DoD)  Digital  Engineering  (DE)  Transformation  initiative,  and  our 
relationship  that  we  have  fostered  with  National  Aeronautics  and  Space  Administration  (NASA) 
Jet  Propulsion  Laboratory  (JPL). 
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1  Introduction 


The  SERC  team  has  conducted  five  working  sessions,  one  special  session  and  19  virtual 
meetings  with  the  United  States  (US)  Army  RDECOM-ARDEC  in  Picatinny,  NJ  to  discuss  the 
needs  and  scenarios  for  a  System  Engineering  (SE)  transformation  enabled  by  evolving  model¬ 
centric  engineering  (MCE)  technologies  and  methods.  Early  meeting  with  ARDEC  covered  their 
prioritization  of  key  areas  to  initiate  such  a  transformation.  We  also  discussed  research  needs 
characterized  as  five  related  tasks  in  the  larger  context  of  ARDEC's  vision  for  an  Armament 
Virtual  Collaboratory  Environment  (AVCE)  integrated  Model  Based  Environment  (iMBE).  We 
refined  those  needs  into  sub-team-related  research  use  cases  that  map  to  ARDEC's  four  of  five 
challenge  area.  ARDEC  is  also  working  with  their  own  Integrated  Product  Teams  (IPTs)  on  some 
of  these  challenge  areas.  We  are  also  fostering  bi-directional  sharing  of  research  interests  and 
results  with  our  US  Navy  Naval  Air  Command  (NAVAIR)  sponsors  who  attended  our  working 
session  in  January  2017.  Finally,  we  are  collaborating  in  several  MCE-related  efforts  to  provide 
the  opportunity  to  leverage  and  share  with  the  Open  Collaboration  Group  for  MBSE  and 
OpenMBEE,  Semantic  Technologies  for  Systems  Engineering  (ST4SE)  initiative,  DoD  Digital 
Engineering  Transformation  Initiative,  the  Aerospace  Industry  Association  (AIA)  on  Concept  of 
Operations  (CONOPS)  for  Government  and  Industry  collaboration  through  MBSE  and  the 
National  Defense  Industry  Association  (NDIA)  Modeling  and  Simulation  group  who  are 
coordinating  working  groups  to  investigate  approaches  for  using  Digital  Models  for  competitive 
down  select. 


1.1  Armament  Virtual  Collaboratory  Environment  Vision 

The  AVCE  iMBE  vision  portrayed  by  ARDEC  reflects  on  their  understanding  of  the  research 
needed  to  advance  to  a  future  state  of  their  integrated  modeling  environment.  There  are  many 
enablers  that  relate  to  characteristics  of  a  holistic  approach  that  aligns  with  their  vision  such  as 
(this  list  is  not  exhaustive,  but  represents  advances  in  use  today): 

■  Mission-level  simulations  that  are  being  integrated  with  system  of  system  (SoS)  and 
system  simulation  that  increasingly  interoperate  with  distributed  interactive  simulation 
capabilities,  augmented  virtual  reality,  and  gaming  technology 

■  Computer-aided  Design  (CAD),  behavioral  techniques,  physics-based/engineering 
simulations,  decision  analytics,  Computer-aided  Manufacturing  (CAM),  system 
architecting,  prototyping,  embedded  in  a  knowledge  management  environment 

■  Enabling  collaborative  environments  by  leveraging  social  media  technologies  and 
operational  metaphors  in  an  engineering  context 

■  Multidisciplinary  Design,  Analysis  and  Optimization  (MDAO)  for  trade  study  analyses 
through  more  systematic  design  of  experiments  allows  engineers  to  make  many  more 
excursions  through  both  the  problem  and  the  design  spaces 

■  Engineering  affordability  analysis,  which  is  a  risk-based  approach  that  could  be  used  to 
significantly  reduce  physical  tests  by  focusing  on  those  system  uses  that  have  the  most 
uncertainty  about  margins  of  performance 

■  Decision  analysis  framework 
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■  Risk  modeling  and  Bayesian-relevant  analysis 

■  Platform-based  approaches  with  virtual  integration 

■  Pattern-based  modeling  based  on  ontologies  with  model  transformation  and  analysis 

■  Domain-specific  modeling  languages 

■  Set-based  design  for  more  concurrent  engineering  and  to  keep  design  options  open 
longer 

■  Modeling  and  simulation  of  manufacturing  and  possibly  early  prototyping 

■  Explosion  of  interactive  visualization,  which  we  will  need  as  we  have  a  "sea"  of  data  and 
information  derived  from  a  "sea"  of  models  with  HPC  computing  capabilities 

The  SERC's  research  with  NAVAIR  Systems  Engineering  Transformation  (SET)  has  provided 
considerable  insights  into  the  challenges  associated  with  MCE  [22].  That  research  suggests  that 
there  is  no  one  instantiation  of  MCE.  Each  organization  will  have  its  own  instantiation  of  its 
"Digital  System  Model"  or  Single  Source  of  Truth  (SST).  A  digital  system  model  will  increasingly 
support  the  integration  of  multi-domain  and  multi-physics  models,  and  provides  for  a  method 
for  model  integrity  for  ensuring  trust  in  models  and  simulation,  and  including  three  critical 
items: 

1.  Cross-domain  model  integration,  and  the  associated  methodologies,  which  will  also 
require  and  contribute  collaboration 

2.  Technologies  to  establish  and  quantify  model  integrity  (trust  in  model  and  simulation 
predictions) 

3.  High  Performance  Computing  (HPC),  which  enables  1  and  2 

The  SST  is  an  enabler  for  cross-domain  interoperability  needed  for  multidisciplinary  design, 
analysis  and  optimization  (MDAO)  for  problem  and  design  space  exploration.  The  SST  requires 
that  all  information  used  to  assess  performance  is  semantically  consistent  with  MCE 
technologies  and  methods  used  for  assuring  integrity  and  the  orchestrated  workflow  is  data- 
driven  (not  process  driven).  SST  provides  the  basis  for  shared-data  and  a  basis  for  real-time 
collaboration. 

As  a  result  of  the  NAVAIR  research  findings  the  Deputy  Assistant  Secretary  of  Defense  (DASD) 
has  initiated  a  Digital  Engineering  strategy.  ARDEC  and  NAVAIR  are  both  participating  in  this 
initiative.  In  addition,  the  SERC  leadership  confirmed  and  recommended  that  complementary 
research  results  can  be  shared  across  these  research  tasks.  To  the  degree  possible  we  are 
synergistically  leveraging  research  completed  or  underway  related  to  NAVAIR  under  SERC  RT- 
157,  RT-170,  and  RT-176  that  includes  other  research  collaborators  Georgia  Tech,  University  of 
Maryland,  and  the  Naval  Postgraduate  School  (NPS). 


1.2  Objectives 

The  critical  items  gleaned  from  the  ARDEC  needs  and  our  prior  research  resulted  in  the 
following  set  of  proposed  tasks: 

■  Task  1:  Framework/architecture  of  development  and  collaboration  environment  that 
support  cross-domain  integration  of  models  to  address  the  heterogeneity  of  the  various 
tools  and  environments 
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■  Task  2:  Formalization  of  an  information  model  for  ARDEC-relevant  domains  to  support 
capturing  and  sharing  of  data 

■  Task  3:  Technology  and  domain-relevant  modeling  methodologies 

■  Task  4:  Demonstrations  in  the  context  of  ARDEC-relevant  Challenge  Areas  relevant  to 
Tasks  1,  2,  3  &  5 

■  Task  5:  System  Engineering  Transformation  Roadmap  to  roll  out  capabilities  addressing 
all  five  perspectives  in  parallel: 

o  Technologies  and  infrastructure 
o  Methodologies  and  processes 

o  People,  training,  competencies  and  framework  viewpoints  and  interfaces 
o  Operational  &  contractual  paradigms  for  transformed  interactions  with  industry 
o  Governance 

We  initially  separated  the  five  tasks  into  subtasks  that  provide  better  mapping  to  research 
expertise,  but  these  are  now  defined  as  linked  use  cases,  which  are  summarized  in  more  detail 
in  Section  2.  These  objectives  underlying  these  tasks  align  with  the  theme  that  were  presented 
at  the  NASA/JPL  Symposium  and  Workshop  on  Model  Based  Systems  Engineering  held  from 
January  25-27,  2017  at  NASA/JPL  in  Pasadena  California.  The  event  brought  together 
practitioners  and  leaders  in  MCE/MBSE  to  share  information  and  ideas  about  the  state  of 
practice,  challenges,  recommendations,  and  future  directions  and  strategies.  Our  ARDEC  and 
NAVAIR  sponsors  were  present  at  this  event  and  should  be  able  to  resonate  with  their  guidance 
on  the  direction  of  our  research. 


1.3  Scope 

In  the  initial  phase  of  the  joint  effort  from  August  2016  to  August  2017,  the  SERC  research 
should  support  ARDEC  interests  to: 

■  Streamline  the  process  for  using  models,  which  is  often  done  only  in  a  relatively  few 
areas  ("pocket"  as  characterized  by  ARDEC) 

■  Understand  the  requirements  for  the  AVCE  conceptually  at  the  stage  of  a  Systems 
Requirement  Review  (SRR);  this  is  not  the  requirements  for  a  target  system,  rather 
these  are  the  requirements  for  a  system  (of  systems)  for  designing  future  ARDEC 
systems  (i.e.,  AVCE  iMBE) 

■  Understand  the  relationships  between  Systems  Engineering  (SE)  activities  and  the 
decision  framework  (related  to  Dr.  Matt  Cilli's  dissertation  [41]);  this  is  related  to  the 
'digital  thread  wheel'  that  can  show  how  to  leverage  analysis  in  each  of  the  areas  to 
develop  a  digital  thread  to  support  repeatable  analysis,  where  a  "fully"  integrated 
operational  analysis  is  missing  currently. 

The  challenge  areas  continue  to  undergo  definition,  refinement  and  alignment.  Four  relevant 
challenges  areas  for  RT-168  as  characterized  by  ARDEC  are: 

■  Challenge  #1:  Taking  existing  ARDEC  models  and  combining  them  to  form  dynamic 
models  at  the  system  level,  and  to  explore  MDAO;  this  will  help  understand  how  the 
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models  interact  with  each  other  and  should  allow  for  use  of  existing  models  to  compose 
and  solve  the  problem 

o  Refinement  by  ARDEC  Integrated  Product  Team  (IPT)  as  presented  at  working 
session  #2:  Develop  an  integrated,  cross  domain,  dynamic  model  of  a  system  to 
assess  its  ability  to  achieve  a  specific  operational  scenario 
o  IPT  Lead:  Rich  Swanson 
o  Status:  completed  at  least  initial  concepts  [9] 

■  Challenge  #2:  Trying  to  understand  the  more  holistic  process  of  solving  a  problem, 
including  the  people  who  are  involved  from  more  of  a  Concept  of  Operation  (CONOPS) 
enabled  by  gaming  technologies,  and  mission-level  modeling  and  simulation  that  can 
ultimately  feed  information  to  a  framework  refined  by  Challenge  #1 

■  Challenge  #3:  The  focus  here  is  on  the  data,  and  how  it  propagates  throughout  the 
lifecycle  and  be  able  to  use  standard-based  and  tool  neutral  technologies  and  methods 
to  "integrate"  modeling  in  analysis  that  are  often  heterogeneous  and  disparate 

o  This  includes  how  the  data  or  metadata  underlying  those  disparate  modeling 
technologies  and  methods  can  be  bi-directionally  linked 
o  Specifically  concerned  with  design  tools  (e.g.,  3D  CAD,  software  development, 

electrical  CAD,  etc.)  that  integrates  with  analysis  tools  (Prism,  IMO,  MagicDraw  etc.) 
that  usually  inputs  design  data  and  produce  analysis  data  and  results,  all  of  which 
needs  to  be  stored  and  managed 
o  IPT  Lead:  John  Campbell 

■  Challenge  #5:  This  is  a  new  challenge  area  defined  in  early  January  of  2017  to  integrate 
crossdomain  models  (SysML  model,  Engineering  Models,  Performance  Models,  Cost 
Models,  etc.)  with  decision  support  model  based  on  Armament  Analytics  Multiple 
Objective  Decision  Analysis  (AAMODAT)  while  executing  Integrated  Systems  Engineering 
Decision  Management  (ISEDM)  process 

o  IPT  Lead:  Matt  Cilli 

This  concept  for  these  four  challenges  as  shown  in  Figure  1,  provides  a  simplified  perspective  on 
the  elements  in  a  more  "traditional  systems  engineering"  perspective.  We  notionally  define: 

■  Concept  of  Operations  (CONOPS)  derived  from  simulation  and  gaming  technologies 

■  "What"  we  want  -  requirements  and  constraints 

■  "How"  (1  or  more)  -  designs  to  achieve  the  "What" 

■  "How  well"  (usually  many)  to  assess  the  "How"  using  analysis,  testing,  reviews  and 
assessing  how  the  design  satisfies  the  requirements,  given  the  constraints  to  achieve  the 
mission  concept 

■  The  underlying  Information  Model  links  the  data  or  metadata  from  many  different 
domains 

■  The  Decision  Framework,  we  believe  can  demonstrate  how  data  from  the  information 
model  can  be  used  to  populate  the  Decision  Framework  in  the  form  of  the 
implementation  of  AAMODAT  with  potential  refinements  and  extensions  supporting  a 
method  to  determine  the  Key  Performance  Parameters  of  the  various  stakeholders. 
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Challenges 
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Figure  1.  Context  of  System  Engineering  of  Challenge  Areas 

MCE  is  enabled  by  computational  technologies  that  now  provide  a  means  for  using  modeling 
and  simulation  in  a  transformed  approach  to  systems  engineering.  A  key  problem  is  that  most 
of  these  technologies  are  not  integrated  currently  (and  many  may  never  be).  The  challenge  area 
presentations  at  the  January  2017  working  session  given  by  ARDEC  confirmed  this  cross-domain 
tool  integration  is  a  challenging  problem.  This  was  further  acknowledged  in  various  talks  at  the 
NASA/JPL  Symposium  and  Workshop  on  Model  Based  System  Engineering  held  January  25-27, 
2017.  Therefore,  we  are  interested  in  an  approach  that  leverages  tool-to-tool  integrations 
where  feasible,  but  the  research  is  targeted  on  approaches  to  using  data  interoperability  as  a 
means  (or  surrogate)  for  accomplishing  integration,  when  tool-to-tool  integration  is  not  feasible 
or  cost-effective.  This  is  challenge  area  #3  that  we  proposed.  We  plan  to  do  research  in  the 
other  two  areas  of  Mission  and  Systems  and  understand  the  flow  of  information  needed  to  be 
linked  between  them,  and  characterize  those  linkages  in  an  Information  Model.  Our  research 
efforts  have  made  progress  in  this  area  that  includes  the  development  of  an  evolving 
Integration  and  Interoperability  Framework  (lolF),  which  has  been  demonstrated  to  ARDEC  at 
both  working  sessions  and  bi-weekly  virtual  events. 

The  new  challenge  area  #5  is  being  coordinated  with  ARDEC's  Dr.  Matt  Cilli  who  believes  that 
information  can  be  captured  to  drive  the  Decision  Support  Model  Construct  [41]  (referred  to  as 
Decision  Framework)  in  the  AAMODAT  tool  developed  and  being  evolved  by  Cliff  Marini.  Other 
research  has  provided  evidence  that  semantic  technologies  (including  ontologies)  may  support 
this  belief  of  Dr.  Chill.  We  believe  Decision  Framework  with  AAMODAT  implementation  serves 
many  purposes  and  benefits: 
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■  Provides  senior  management  and  program  managers  with  visual  representations  of  key 
tradeoff  defined  in  terms  of  Key  Performance  Parameters  (KPPs)  such  as  Performance, 
Cost,  Time  and  Risk 

■  As  shown  in  Figure  2,  scatterplot  shows  in  a  single  chart  how  system  level  alternatives 
respond  in  multiple  dimensions  of  stakeholder  value 

■  Assessment  Flow  Diagrams  (AFDs)  trace  the  relationships  between  physical  means, 
intermediate  measures,  and  fundamental  objectives 

■  Provides  methodological  guidance  for  identifying  KPPs 

■  Can  be  used  with  uncertainty  analysis  as  a  measure  for  understanding  maturing  design 

■  Enables  bi-directional  analysis  throughout  lifecycle 
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Figure  2.  Decision  Support  Model  Construct 

The  ARDEC  leadership  and  SERC  team  agreed  on  the  challenge  area  scenarios  for  using  some 
examples  related  to  counter  unmanned  aerial  systems  case  study.  The  team  has  constructed 
several  artificial  UAS  scenarios  (use  cases)  and  evolving  scenario  variants  that  demonstrate 
methods  to  address  many  of  the  cross-cutting  concerns  from  CONOPS,  mission  and  system 
engineering.  Mission-level  scenarios  have  been  created  and  demonstrated  using  four  different 
modeling  and  simulation  capabilities  ranging  from  low-cost  and  low-fidelity  to  high-cost  and 
high-fidelity. 

We  fully  assume  that  there  will  be  practical  limitations  to  fully  automating  the  concept 
discussed  in  this  section,  however,  given  the  objectives,  a  value  and  unique  contribution 
proposed  by  this  research  is  on  the  appropriate  system  (and  SoS)  methodological  guidance  in 
the  context  of  specific  technologies.  Our  sponsor  has  stated  that  they  believe  the  efforts  to 


Report  No.  $ERC-2017-TR-n0 


6 


Date:  August  8,  2017 


Contract  No.  HQ0034-13-D-0004 


date  have  helped  ARDEC  in  making  decisions  on  approaches  to  the  development  of 
requirements  and  architectures  for  AVCE  iMBE. 

We  have  obtained  and  use  academic  licenses  for  some  of  the  most  powerful  commercial  tools 
in  order  to  address  research  questions  in  the  context  of  these  types  of  tools;  these  are  the 
types  of  tools  used  by  both  ARDEC  and  industry.  This  approach  also  addresses  some 
organizational  and  domain-specific  concerns.  Through  digital  means  we  can  now  also  encode 
historical  knowledge  in  reference  models,  model  patterns  to  embed  methodological  guidance 
to  support  continuous  orchestration  of  analysis  through  new  modeling  metrics,  and  automated 
workflow  to  accelerate  concepts  to  prototypes,  deployment  and  foster  event-driven 
collaboration.  Therefore,  the  deliverables  include  reports,  demonstrations,  meetings,  meeting 
notes,  and  examples  of  models  without  violating  any  of  the  academic  licensing  guidelines. 


1.4  Organization  of  Document 

Section  1  provides  an  overview  of  the  context  for  the  needed  research,  objectives,  scope  and 
organization  of  this  report. 

Section  2  provides  a  summary  of  the  current  set  of  research  use  cases,  our  Phase  I  efforts, 
status,  and  recommendations  based  on  our  increased  understanding  of  the  research  objectives. 
For  purposes  of  understanding  the  evolving  efforts  and  status,  the  overview  presented  in 
Section  2  should  provide  that  level  of  information. 

Part  II  describes  the  detailed  research  use  cases. 

Section  3  discusses  the  concept  of  the  Information  Model  underlying  the  AVCE;  the 
fundamental  purpose  is  to  provide  a  means  to  link  information  and  metadata  from  disparate 
sources  across  the  various  domains. 

Section  4  describes  the  concept  for  researching  the  use  of  Graphical  CONOPS,  including  the 
potential  relationships  with  the  Early  Synthetic  Prototyping  under  research  at  University  of 
Southern  California  (USC)  Institute  for  Creative  Technologies  (ICT). 

Section  5  describes  research  into  the  use  of  mission  and  system  modeling  and  simulation,  and 
its  relationships  to  graphical  CONOPS  and  MDAO. 

Section  6  discusses  modeling  methodologies,  including  examples  and  demonstrations  created 
to  illustrate  mission,  system,  enterprise  and  reference  models,  including  example  and  methods 
for  MDAO. 

Section  7  provides  an  overview  of  the  approach  for  developing  system  models  using  Model 
Based  System  Engineering  (MBSE),  but  more  importantly  for  understanding  the  ways  to  linking 
MBSE  models  through  the  MCE  toolchain  as  it  relates  to  requirements  for  AVCE. 

Section  8  provides  an  overview  of  the  approach  for  relating  system  models  using  MBSE,  Model 
Based  Engineering  (MBE),  but  more  importantly  for  understanding  the  ways  to  link  MBE  models 
through  the  MCE  toolchain  as  it  relates  to  requirements  for  AVCE.  Some  of  the  details  of  the 
Courter  UAS  are  covered  in  this  use  case,  and  a  new  section  on  Automated  Concurrent 
Engineering. 
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Section  9  discusses  the  research  approach  to  leverage  information  captured  through  all  of  the 
phases  and  types  of  modeling  into  the  information  model  to  systematically  populate  the 
Decision  Framework  as  implemented  currently  in  AAOMDAT. 

Section  10  discusses  potential  contributions  of  modeling  to  support  verification  and  validation 
(V&V). 

Section  11  is  a  use  case  to  develop  and  assess  the  operational  elements  of  the  entire 
framework  in  the  context  of  a  Chief  Engineer  Role. 

Section  12  describe  tradeoff  analysis  of  technologies  for  integration  or  interoperability  as  a  way 
for  representing  and  analyzing  the  architecture  trades  for  the  requirements  of  AVCE.  In 
addition,  this  section  reflects  on  some  of  the  most  advanced  integrated  modeling  environment 
identified  through  the  NAVAIR  related  SERC  research  tasks.  This  task  has  been  extended  to 
consider  Windchill,  which  builds  off  of  a  prior  SERC  RT-152,  other  commercial  tool  examples, 
and  involvement  with  the  Open  Collaboration  Group  for  MBSE  and  OpenMBEE. 

Section  13  discusses  the  use  of  Semantic  Web  Technologies  applied  to  AAMODAT  for  the  newly 
defined  challenge  area  #5. 

Section  14  provides  a  new  use  case  for  assessing  the  AVCE  iMBE  requirements  and  model. 

Section  15  provides  a  description  of  some  of  the  SERC  research  synergies  that  are  relevant  to 
the  ARDEC  research  objectives. 

Section  16  provides  a  summary  of  Part  II. 
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2  In-Process  Summary 


This  section  provides  context  into  the  scope  and  approach  to  this  research.  The  research 
continues  to  evolve  as  we  have  more  in-depth  discussions  and  demonstrations  with  ARDEC 
about  the  research  and  potential  benefits.  For  example,  Eddie  Bauer's  briefing  for  a  Digital 
Engineering  Working  Group  meeting  stated:  "Research  in  Data  Ontology/Information  Model 
using  semantic  web  ontologies  is  promising  and  could  support  model  and  simulation 
integration."  [9] 

Some  of  the  research  results  are  emerging  as  elements  of  ARDEC's  concept  and  architecture  for 
AVCE  iMBE.  There  is  understanding  that  semantic  technologies  provide  potential  to  better 
understand  the  detailed  information  model  in  a  semantically  precise  way  and  enables 
underlying  computation  capabilities  to  automate  reasoning  about  systems  engineering  tasks.  In 
addition,  the  semantic  precision  and  cross-domain  linkages  of  information  enables  more 
computational  analytics  about  consistency,  completeness  and  well-formed  of  captured 
information. 

We  are  using  a  Model  Based  System  Engineering  (MBSE)  approach  to  model  our  project,  and 
also  to  assist  ARDEC  in  assessing  their  AVCE  iMBE  models.  We  started  to  elaborate  the  research 
tasks  using  high-level  use  cases  as  shown  in  Figure  3,  relating  those  use  case,  and  associating 
the  use  case  with  the  stakeholders  involved  in  the  research.  The  relationships  between 
stakeholders  and  use  cases  reflects  on  the  interactions  and  dependencies  between  the  team's 
research. 
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2.1  ARDEC  Challenge  Area  #1  Preliminary  Findings 

The  status  presented  at  the  second  working  session  given  by  Rich  Swanson  who  is  leading  the 
ARDEC  team  on  the  challenge  area  #1  focused  on  integrated,  cross  domain,  dynamic  model  of  a 
system  to  assess  its  ability  to  achieve  a  specific  operational  scenario.  The  efforts  to  date  are 
identifying  the  linkage  between  those  domain  areas  by  executing  a  conceptual  scenario  of 
counter  UAS  requirements  to  help  inform  the  team  about  gaps  and  challenges  associated  with 
requirements  needed  for  the  AVCE  iMBE  requirements  and  architecture.  The  lessons  learned 
confirms  that,  while  technically  feasible,  there  are  challenges  in  achieving  cross  domain  models 
integration  to  facilitate  sharing  of  relevant  data  between  specialties  in  order  to  assess 
performance  within  the  scenario. 

The  following  provides  a  few  of  challenges  and  concerns  derived  from  the  briefing  material  and 
presentation  provided  by  the  ARDEC  challenge  area  #1  lead  (non-exhaustive): 

■  Automation  leading  to  a  lack  of  applied  subject  matter  expertise  and  granularity  of 
assessment  within  each  step  of  the  analysis,  can  lead  to  incorrect  analysis  and 
assumptions 

o  People  have  access  to  tools  and  data,  but  may  not  understand  the  methods  to 
effectively  use  the  tools 

o  Tools  may  not  have  been  created  with  adequate  checks  such  as  input  data  validity; 
again  this  relates  to  methods  and  types  of  checks  that  could  be  performed  in  an 
information  model  through  semantic  web  technology  (see  NASA/JPL  example) 
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■  Working  across  domains,  both  from  a  technical  and  socio-technical  perspective  is 
challenging;  if  the  tools  worked  better  across  domains,  would  this  help  with  the  socio- 
technical  ("people") 

o  Shows  the  need  for  both  the  iMBE  objective  framework  (see  new  challenge  area  #5) 
o  There  are  concerns  that  integrated  modeling  will  not  improve  the  timeline 

dramatically,  because  integration  of  models  takes  time,  especially  when  emerging 
scenarios  or  technologies  are  included 
o  Simulations  can  take  long  time  to  run,  but  uncertain  if  simulations  are 
"structured/programmed"  to  leverage  high  performance  computing 

■  Cost  of  integration  and  automation  must  be  weighed  against  value  it  can  provide. 
Consider: 

o  Integration  of  models  versus  integration  derived  through  interoperability  using 
standardized  (format)  data  (i.e.,  data/information  model) 
o  Digital  thread  provided  by  traceability  between  models  versus  single  source  of  truth 

These  findings  were  also  characterized  in  a  different  way  in  a  briefing  given  by  Eddie  Bauer  at 
the  Digital  Engineering  Working  Group  that  is  approved  for  public  distribution  [9]: 

■  Culture 

o  Uncovered  lack  of  understanding  across  Integrated  Product  Team  (IPT)  specialties 
for  the  detail,  and  sometimes  value  that  other  IPT  members  provide 
o  Lack  of  trust  that  data/models  will  be  used  appropriately 

•  NAVAIR  generally  refers  to  this  as  Model  Integrity  (trust  in  the 
models/simulation  results) 

o  SME  involvement  must  never  be  overlooked  as  integrated  models  can  easily  lead  to 
incorrect  analyses  and  assumptions 

o  Do  the  SME's  want  to  be  integrated?  Need  to  better  understand  value  of  Dynamic 
System  models 

■  Model  Integration 

o  Technically  possible 
o  Domain  understanding  is  very  important 

o  Physically  passing  data  to  appropriate  SMEs  is  not  the  time-sink;  it  is  developing  new 
or  modifying  existing  models  for  a  new  scenario  -  assessing  and  validating  results  is  a 
concern 

o  Dynamic  Model  Complexity  =  Greater  Run  Time  and  Need  for  High  Performance 
Computing 

■  Authoritative  Source  of  Truth 

o  Need  a  common  library  of  models/integrated  models 

o  Requires  rigor  in  documenting  M&S  details  that  are  normally  in  the  analyst's  head 
o  Results  of  analyses  need  to  retain  input  from  the  SMEs  involved  in  its  development 
and  execution 

Many  of  these  concerns  align  with  the  findings  from  the  NAVAR  research  as  summarized  in 
technical  reports  for  RT-118/141/157  [22]  [23]  [25]. 
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2.2  Use  Case  Summary 

This  section  provides  a  high-level  summary  of  each  use  case  and  recent  results.  Part  II  (starting 
with  Section  3)  of  this  report  provides  additional  details  on  each  use  case  (UC).  As  shown  in 
Figure  3,  there  is  considerable  emphasis  on  understanding  many  of  the  cross-domain 
dependencies  of  the  research  use  cases,  and  understanding  the  methods  that  must  be  used  to 
guide  the  production  of  this  information  across  the  various  domains  and  lifecycle  phases. 

We  are  developing  an  Integrating  and  Interoperability  Framework  (lolF)  as  part  of  UC09  as 
shown  in  Figure  4.  We  are  working  with  other  use  case  teams  to  provide  a  demonstrations  of 
the  Decision  Framework  (UC06)  enabled  by  semantic  technology  (UCOO).  We  envision  using 
semantic  web  technologies  (SWT)  in  the  context  of  the  Decision  Layer  process  with  AAMODAT 
(UC10)  highlighted  in  orange  oval  to  be  in  this  part  of  the  concept.  In  collaboration  with  NAVAIR 
and  NASA/JPL,  we  would  also  like  to  bring  in  the  Integrated  Model  Centric  Engineering  (IMCE) 
ontologies  [91]  for  systems  engineering.  We  are  considering  using  tool-to-tool  integration  as 
discussed  in  UC09,  Data  Acquisition  and  Aggregation  in  research  to  integrate  Graphical  CONOPS 
(UC01),  and  Mission  and  System  Operational  Capabilities  (UC02). 


Figure  4.  Integrating  and  Interoperability  Framework  (lolF) 

00.  Develop  Information  Model.  This  information  model  characterizes  the  underlying 
information  and  relationships  to  "everything"  that  might  need  to  be  produced  by  the  tools 
of  AVCE,  although  we  are  using  tools  available  to  our  Stevens  laboratory.  This  has 
significant  relationships  to  challenge  #3  and  #5.  We  are  using  the  SWT  language  the  Web 
Ontology  Language  (OWL)  [179]  as  the  primary  means  for  characterizing  the  information 
model  across  many  of  the  use  cases.  As  reflected  in  Figure  1,  the  challenge  is  to 
characterize  this  information  for  each  of  the  various  domains,  including  requirements,  risks, 
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designs  (e.g.,  electrical,  mechanical,  etc.),  and  analyses.  This  reflects  why  there  are  so  many 
associations  from  the  other  use  cases.  In  addition,  we  (including  our  ARDEC  IPT) 
fundamentally  believe  that  it  is  technical  feasible  to  capture  this  information  and  provide  it 
as  input  to  the  Decision  Framework  (UC06).  The  research  demonstrated  the  use  of  SWT  to 
demonstrate  the  concept  to  both  characterize  the  data  and  information  as  well  as  rules, 
and  query  language  for  processing  and  data  exchange  during  working  session  five  (5). 

■  Several  demonstrations  have  illustrated  the  feasibility  of  this  concept  in  both  working 
sessions  and  webinar  sessions.  Challenge  area  #5  has  been  defined  and  the 
prioritization  of  the  information  model  will  align  with  the  objective  to  characterize  the 
information  and  rules  associated  with  inputs  to  AAMODAT;  as  such,  this  is  now  defined 
as  a  new  user  case  UC10. 

■  The  SWT  is  being  architecturally  represented  in  the  Integrating  and  Interoperability 
Framework  (lolF)  as  part  of  UC09,  which  was  also  demonstrated. 

01.  Research  Graphical  CONOPS.  Investigate  the  use  of  Graphical  CONOPS  technologies  such  as 
gaming  environments.  The  team  has  created  demonstrations  using  the  Unity  gaming 
engine  [170]  for  simulating  two  autonomous  UAS  interacting  in  an  environment.  Our 
research  collaborators  USC/ICT  have  been  evolving  a  technology  called  Early  Synthetic 
Prototyping  (ESP).  We  are  fundamentally  interested  understanding  if  there  is  an  underlying 
metamodel  of  the  information  that  can  be  captured,  regardless  of  the  domain,  and  the 
methods  that  would  be  used  to  ensure  that  information  is  fully  captured.  This  information 
would  be  mapped  to  the  Information  Model  (UC00)  and  be  provided  as  input  to  UC02.  In 
addition,  we  are  interested  in  how  the  parameters  of  simulation  entities  can  be  used  in 
MDAO  (UC03). 

■  The  metamodel  provided  by  ICT  represents  information  and  metrics  captured  while 
observing  the  users  of  the  gaming  technologies;  processing  this  information  in  real-time 
has  shown  to  be  difficult,  but  having  this  type  of  information  stored  in  SWT  (UC00)  could 
enable  better  and  real-time  analytics,  which  has  been  stated  as  a  desire  by  our  ARDEC 
sponsor. 

■  There  have  been  nine  updates  to  the  graphical  CONOPS,  which  provides  two  types  of 
missions  for  red/blue  surveillance  missions  for  autonomous  quadcopters.  The  updated 
simulations  include  more  realistic  battery  and  flight  models  (UC05),  and  current 
research  is  using  MDAO  (UC03)  for  this  level  of  the  mission  analysis. 

02.  Research  Mission  and  System  Operational  Capabilities.  Investigate  the  methodological  and 
relevant  technologies  for  mapping  the  Graphical  CONOPS  into  Mission  and  System 
modeling  and  simulation  capabilities.  The  current  research  involves  the  use  of  VT  MAK 
[103]  and  other  2D  modeling  and  simulation  environments  for  distributed  simulations.  We 
envision  that  information  from  UC01  would  provide  parameter  information  that  can  be 
refined  or  expanded.  Therefore,  like  UC01,  we  want  to  understand  the  underlying 
information  (e.g.,  metamodel)  that  would  be  mapped  to  the  Information  Model  (UC00), 
and  the  associated  methods  for  how  to  develop  models  at  this  level.  This  use  case  is  also 
researching  the  relationships  of  these  simulation  models  and  system  models  in  languages 
such  as  SysML. 
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■  We  have  created  a  simple  ontology  as  the  basis  to  demonstrate  information  sharing 
through  SWT  to  illustrate  transfer  of  information  through  the  SWT  components  of  the 
lolF.  The  demonstration  also  illustrated  the  use  of  triple  stores  and  SPARQL  [181] 
queries  to  store,  extract  or  transform  data  in  the  SWT.  The  next  planned  demonstration 
will  use  these  lolF  capabilities  to  transfer  data  between  the  Graphical  CONOPS 
simulation  and  low  fidelity  mission-level  analysis  on  a  2D  plane  with  spatial  positions  of 
entities. 

■  This  use  case  is  also  researching  the  relationships  of  these  simulation  models  and 
system  models  in  languages  such  as  SysML. 

03.  Research  MDAO.  Investigate  the  methods  to  trace  capabilities  to  the  relevant  design 
disciplines  and  perform  cross-domain  analyses  through  MDAO  for  problem  and  design 
tradespace  analyses.  In  addition,  to  characterizing  elements  of  the  framework,  cross¬ 
domain  relationships,  but  also  characterize  the  methods  used  to  support  MDAO  in  a  tool 
independent  manner  (we  obtained  academic  licenses  for  ModelCenter,  because  we  know 
that  ARDEC  uses  that  tool;  these  license  can  be  used  to  provide  examples,  but  not 
contribute  to  any  ARDEC-specific  work). 

■  Recent  updates  of  UAV  model  using  MDAO  workflows  in  ModelCenter  show  more 
realistic  results  in  terms  of  weight  and  size,  including  use  of  Computational  Fluid 
Dynamics,  results  of  Design  of  Experiments  (DOE)  for  range  vs.  cruise  altitude  vs 
wingspan,  and  a  Pareto  frontier  for  range,  payload,  and  endurance  as  KPPs,  new 
visualizations  provided  by  version  12  of  ModelCenter 

■  Another  model  that  was  used  Phoenix  Integration  MBSE  Analyzer  to  integrate 
MagicDraw  SysML  with  ModelCenter 

■  Current  efforts  are: 

o  Researching  use  of  ModelCenter/MDAO  to  the  Graphical  CONOPS  (UC01) 
o  Investigating  the  use  of  using  MBSE  Analyzer/MagicDraw  SysML  with  ModelCenter 
to  formalize  the  Assessment  Flow  Diagrams  (AFD)  for  the  Decision  Framework 
(UC06) 

04.  Create  System  Models.  This  applies  MBSE  to  the  case  study  examples  and  looks  at  how 
metamodels  or  metadata  is  represented  in  the  Information  Model  (UC00)  to  provide 
traceability  through  the  other  forms  of  modeling  for  UC01,  UC02,  UC03  and  UC05.  This  use 
case  is  developing  different  variants  of  UAS  system  models  at  both  the  system  and  mission 
level. 

■  Demonstrations  include  the  use  of  the  OpenMBEE  Model  Development  Kit  (MDK) 
DocGen  to  a  number  of  models  including  the  AVCE  iMBE  and  Rotocopter  UAV 

■  We  have  an  evolving  SysML  model  for  the  RT-168  lolF  framework  (UC09)  to  formalize 
the  architecture,  which  has  been  provided  to  ARDEC 

■  We  are  near  completing  setup  of  the  OpenMBEE  environment,  including  the  Model 
Management  System  (MMS)  and  View  Editor  components  that  have  been  open-sourced 
by  NASA/JPL  at:  http://www.openmbee.org/  this  is  planned  to  be  integrated  with  the 
lolF  framework 
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■  We  are  trying  to  leverage  work  with  the  SERC  RT-176  led  by  Kristin  Giammarco  to  use 
Monterey  Phoenix  (MP)  for  demonstrating  the  potential  to  perform  early  V&V 
requirements  and  architecture  models  [70].  Currently,  MP  is  a  language,  but  we  believe 
we  can  develop  a  graphical  language  using  SysML  activity  diagram  (maybe  profiled),  and 
then  use  DocGen  to  extract  information  in  order  to  translate  into  MP.  This  tasks  benefits 
ARDEC,  because  RT-176  is  funded  by  NAVAIR. 

05.  Use  Model  Based  Engineering.  This  applies  Model-Based  Engineering  (MBE)  typically 
associated  with  the  different  design  disciplines  (e.g.,  electrical,  mechanical,  controls)  and 
will  focus  on  some  related  research  associated  with  counter  UAS.  Like  UC04,  we  are 
interested  at  how  metamodels  from  these  various  domain  or  metadata  are  represented  in 
the  Information  Model  (UC00)  to  provide  traceability.  It  is  currently  acknowledged  that, 
except  for  a  few  exceptions  there  is  a  gap  in  mapping  from  these  types  of  modeling 
technologies  to  MBSE  models. 

■  Presented  a  session  on  "Representation  Methods,  Model  Frameworks  and  Verification 
Tools  for  CPS  Design"  for  UAS 

■  Current  investigations  include  bringing  MBE  design  information  into  the  SWT  using  an 
architecture  and  prototyping  of  system  simulation  with  semantic  data  exchange;  this 
will  look  at  discipline-specific  ontologies  for  cross-domain  integration  [29] 

06.  Research  Decision  Framework.  As  discussed  in  Section  1.3,  we  have  had  discussions  with 
the  ARDEC  leads,  who  are  intimately  familiar  with  this  framework  and  the  evolving  tool 
called  AAMODAT.  This  use  case  is  now  aligned  with  challenge  area  #5.  Fundamentally,  a  key 
goal  for  UC00  is  to  capture  information  that  can  be  used  to  provide  input  to  the  Decision 
Framework  (UC06).  This  would  provide  senior  leaders  and  program  managers  the  type  of 
information  they  need  to  consider  technology  capability  tradeoff  using  Performance,  Cost 
(Affordability),  Time  (delivery  schedule)  and  Risk.  Fundamentally,  if  a  particular  answer  was 
unacceptable,  using  the  concept  discuss  herein,  we  could  trace  linkages  through  the 
Information  model  back  to  all  other  related  perspectives  on  the  system  (UC01,  UC02,  UC03, 
UC04,  UC05). 

■  We  provided  demonstrations  using  SWT  to  get  example  data  from  DBpedia  (which  is  a 
crowd-source  effort  to  extract  structured  information  from  Wikipedia  and  make  this 
information  available  on  the  Web)  of  a  simple  aircraft  ontology  and  properties  to  show 
semantically  rich  data  extracted  from  DBpedia  using  SWT  tools  (Protege,  OWL  Viz,  RDF) 

■  Investigating  the  use  of  Phoenix  Integration  MBSE  Analyzer  plugin  to  MagicDraw  SysML 
with  ModelCenter  to  formalize  the  Assessment  Flow  Diagrams  (AFD)  for  the  Decision 
Framework  (UC06)  using  an  updated  UAV  case  study  [42] 

■  Working  on  templates  for  different  type  of  objective  hierarchies  (e.g.,  portfolio, 
product) 

07.  Research  Verification  and  Validation  (V&V).  This  use  case  was  not  considered  in  the  original 
plan,  but  MCE  does  provide  some  unique  opportunities  to  be  more  effective  at  contributing 
V&V  evidence  in  early  design.  Rigorously  defined  models  can  directly  support  V&V,  and  this 
could  both  subsume  cost  and  risks.  This  use  case  can  likely  identify  candidate  requirements 
for  AVCE. 
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■  As  discussed  in  UC04,  we  are  trying  to  leverage  work  with  the  SERC  RT-176  led  by  Kristin 
Giammarco  to  use  Monterey  Phoenix  (MP)  for  demonstrating  the  potential  to  perform 
early  V&V  requirements  and  architecture  models  [70]. 

08.  Assess  as  Chief  Engineer  Role.  This  use  case  is  created  so  that  one  of  our  researchers, 
experienced  in  actual  systems  engineering  can  provide  some  level  of  assessment  of  our 
overarching  approach  and  contribute  to  the  requirements  for  AVCE.  We  too  want  to  bring 
as  many  technologies  as  possible  into  our  lab  at  Stevens  in  order  to  assess  the  gaps,  but  are 
also  interesting  in  bring  in  Masters  students  to  using  methods  derived  from  this  research. 

09.  Tradeoff  Analysis  of  Technologies  for  Integration  or  Interoperability.  This  use  case  has  been 
renamed  and  expanded  due  to  information  learned  about  other  technologies  that  provide 
a  means  for  looking  at  alternative  technologies  and  approach  to  support  either  tool 
integration  or  some  type  of  equivalent  interoperability  approaches  that  can  be  used  for 
AVCE.  Specifically,  we  are  looking  at  the  technologies  and  tools  used  by  ARDEC  and  used  in 
the  case  study  to  focus  this  research.  In  addition,  this  tasks  revisits  some  of  the  most 
advanced  tool  integrations  that  have  been  developed  by  NASA/JPL  [59]  [10],  the  DARPA 
META  projects  [8]  [7],  Engineered  Resilient  Systems  [81],  Airbus  [76],  and  generalization  of 
commercial  and  industry  integrated  modeling  environments.  We  added  a  team  member 
assess  Windchill  as  part  of  this  use  case.  We  learned  of  Syndeia  by  Intercax,  and 
coordinated  a  demonstration  with  our  ARDEC  sponsor.  We  have  joined  Open  Collaboration 
Group  for  MBSE  and  OpenMBEE  [132]. 

■  As  discussed  at  the  beginning  of  this  section,  the  lolF  as  shown  in  Figure  4,  brings  a 
number  of  use  cases  together: 

o  The  SWT  is  being  expanded  to  support  interoperability  from  Graphical  CONOPS 
(UC01)  to  Mission-level  simulation  (UC02) 
o  We  are  modeling  this  architectural  framework  (UC04) 

o  We  are  expecting  disciplines  specific  information  to  be  integrated  through  the  SWT 
component  (UC05) 

o  We  expect  this  same  architectural  element  to  be  used  to  support  exacting 
information  to  populate  the  Decision  Framework  (UC06)  and  AAMODAT  (UC10) 
o  We  will  also  look  to  integrate  these  capabilities  with  OpenMBEE 

10.  Challenge  area  #5  has  been  defined  and  the  prioritization  of  the  information  model  will 
align  with  the  objective  to  characterize  the  information  and  rules  associated  with  inputs  to 
AAMODAT.  This  use  case  is  related  to  both  UC00  and  UC06. 

■  We  discussed  how  AAMODAT  is  usually  something  that  happens  early  on  for  ARDEC, 
and  all  over  the  project.  It  has  helped  to  identify  Key  Performance  Parameters  (KPPs)  at 
the  mission  level  and  the  elements  from  the  sub-domains  that  are  relevant  to  those 
KPPs.  'All  requirements  are  tradeable,'  but  looking  at  how  much  they  contribute  to  the 
KPPs,  is  a  different  way  of  thinking. 

■  As  discussed  in  UC09,  we  expect  this  same  architectural  element  to  be  used  to  support 
exacting  information  at  populate  the  Decision  Framework  (UC06) 
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11.  Assess  AVCE  iMBE.  We  were  asked  to  provide  a  more  detailed  analysis  of  the  AVCE  iMBE 
requirements.  We  initially  looked  at  the  requirements,  but  in  attempt  to  do  the  analysis 
started  to  identify  additional  use  cases  not  reflected  in  the  model  as  shown  in  Figure  11. 
ARDEC  then  did  deliver  the  AVCE  iMBE  model,  and  we  developed  a  set  of  View  and 
Viewpoints  for  the  model  to  allow  for  us  of  MDK/DocGen.  While  the  model  is  well 
structure,  the  View  and  Viewpoints  modeling  process  revealed  some  minor  inconsistencies, 
which  we  shared  with  ARDEC.  While  ARDEC  has  finished  the  Systems  Requirement  Review 
(SRR)  for  AVCE  iMBE.  Rick  Dove  joined  the  RT-168  research  team. 

■  Rick  Dove  has  done  some  research  through  the  INCOSE's  Agile  Systems  Engineering  Life 
Cycle  Model  (ASELCM)  project,  and  specifically  in  terms  characterized  by  the  ASELCM 
Pattern  of  Three  Concurrent  Systems.  Rick  will  use  this  context  to  look  at  the  AVCE  iMBE 
model  from  this  three-system  perspective 


2.3  Working  Sessions  and  Sponsor-Supporting  Events 

A  component  of  the  research  and  required  deliverables  are  conducting  working  sessions  that 
inform  the  ARDEC  team  about  progress  against  the  plan.  These  working  session  also  inform  the 
team  about  relevant  information  and  feedback  to  scope  the  deliverables  in  the  context 
appropriate  for  ARDEC;  this  approach  has  been  especially  important  for  working  other  SERC 
research  task,  such  as  with  NAVAIR  given  the  recent  changes  under  SE  transformation.  In 
addition,  NAVAIR  joined  for  the  second  half  day  meeting  for  the  first  working  session,  and  a 
number  of  members  of  the  NAVAIR  team  have  been  attending  working  sessions  and  the  bi¬ 
weekly  meetings. 

■  Working  session  #1:  21,  22-Sep-2016  held  at  ARDEC 

o  The  SERC  team  provided  an  overview  elaborated  from  the  proposal  discussing  an 
approach  to  use  case  study  scenarios  to  address  the  lifecycle  concerns  from 
CONOPS,  mission  and  system  analysis,  using  MDAO  fortradespace  analysis,  Model- 
Based  System  Engineering  linking  to  risk  and  the  decision  framework.  This  was 
presented  in  the  context  of  their  Digital  Thread  concept.  The  SERC  team  also 
discussed  the  potential  synergies  with  NAVAIR  Systems  Engineering  Transformation 
and  the  Digital  Engineering  Strategy  initiative  coordinated  by  Deputy  Assistant 
Secretary  of  Defense  (DASD).  Discussed  the  concept  for  developing  the  ontology 
underlying  the  requirement  manager  (top-level  priority) 

■  Working  session  #2:  10-Jan-2017  held  at  ARDEC 

o  This  session  covered  the  broad  objectives  identified  by  ARDEC,  to: 

•  Discuss  progress  in  research  areas 

•  Share  lessons  learned  from  their  own  efforts  on  Challenge  Areas 

•  Identify  areas  for  enhanced  collaboration 

•  Engage  in  general  model-based  engineering  discussions 

o  A  number  of  presentations  and  demonstrations  from  ARDEC,  SERC,  and  NAVAIR 
were  given  to  inform  the  audience  and  to  stimulate  further  discussions,  including: 

•  Status  of  AVCE-iMBE  Project -ARDEC,  Cliff  Marini 

•  Dynamic  Model  Challenge  Overview  -  ARDEC,  Rich  Swanson 
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•  NAVAIR  SE  Transformation  Overview  -  NAVAIR,  Jaime  Guerrero 

•  Overall  Status  of  RT-168  Transforming  Systems  Engineering  through  Model- 
Centric  Engineering  -  SERC,  Mark  Blackburn 

•  Demonstration:  Graphical  CONOPS  -  SERC,  Roger  Jones 

•  Demonstration:  VT-MAK  Mission  Simulation  -  SERC,  Roger  Blake 

•  Integrated  Mission  Modeling:  Approach  and  Initial  Results  -  SERC,  Paul  Grogan 

•  Demonstration:  Multidisciplinary,  Design,  Analysis  and  Optimization  -  SERC, 
Steven  Hoffenson 

•  Overview  of  Integrated  Model  Based  Engineering  Environment  (iMBE-E)  Data 
Challenges  -  ARDEC,  John  Campbell 

•  Data  Ontology/Information  Model  -  SERC,  Mark  Blackburn 

•  Decision  Framework  Approach  and  AAMODAT,  ARDEC,  Matt  Cilli 

■  Working  session  #3:  30-Mar-2017  held  at  Stevens 
o  ARDEC  AVCE-iMBE  Update,  Cliff  Marini 

o  NAVAIR  Progress  update,  Mark  Blackburn 

o  RT  168  Progress  update,  Mark  Blackburn 

o  Semantic  Web  Technologies  Demo  &  Discussion,  Mary  Bone 

o  Semantic  Web  Technologies  Demo  and  Discussion...  continued 

o  USC  ICT  Research  Presentation,  Edgar  Evangelista 

o  MBE  Tools:  Syndeia,  OpenMBEE,  Jeff  McDonald,  Mark  Blackburn 

o  Mission-level  simulation  using  High  Level  Architecture  (HLA)  Demo,  Roger  Blake, 

Paul  Grogan 

■  Working  session  #4:  13-Jun-2017  held  at  ARDEC 

o  ARDEC  updates,  Christina  Jauregui,  Cliff  Marini,  Greg  Nieradka 
o  OpenMBEE,  Mark  Blackburn 

o  OpenMBEE  MDK/DocGen  for  the  AVCE  model,  Benjamin  Kruse 
o  Sys  ML/M  DAO/M  BSE  Analyzer,  John  Dzielski 
o  MDAO  updates,  Brian  Chell 

o  Graphical  CONOPS  update  and  demonstration,  Roger  J. 

o  Semantic  Technology  for  SE  Working  Group/  NASA/JPL  Integrated  Model  Centric 
Engineering  (IMCE)  Ontologies  and  SWT,  Mark  Backburn,  Mary  Bone 
o  Integration  and  Interoperability  Framework  (lolF)  -  Demonstration,  Roger  B,  Roger  J, 
Paul) 

o  NAVAIR  RT-170/RT-176  updates,  Modeling  for  the  Surrogate  Pilot,  Mark  Blackburn 
o  Requirement  V&V  through  Monterey  Phoenix  (Mark  Blackburn) 

■  Special  Session:  31-July-2017  held  at  Stevens 

o  This  special  session  invited  our  sponsors  from  ARDEC,  NAVAR,  and  DASD(SE),  but 
also  other  organization  Naval  Surface  Warfare  Center,  Digital  Warfare  Office,  and 
MITRE,  and  industry  guests  from  Raytheon  working  on  Semantic  Web  Technologies 
and  Ontologies 

o  Objectives  included:  "Provide  Big  Picture  -  Mental  Model" 
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•  Use  historical  context  of  research  investigating  "the  most  advanced  and  holistic 
approaches  and  technologies  supporting  state-of-the-art  in  Model  Centric 
Engineering"  aka  Digital  Engineering 

•  Summarize  expanse  of  research  thrusts  dating  back  to  initial  NAVAIR  air  research 
in  2013 

•  Discuss  alignment  with  sponsors'  evolving  needs,  transformation,  and  goals  of 
digital  engineering  initiative 

•  Provide  awareness  of  collaborations  with  other  initiatives,  industry,  government, 
academia  &  open  communities 

o  "Past  -  Why"  -  Historical  perspectives  -  How  we  got  here  and  why 
o  "Present  -  What"  -  Aligning  the  research  gaps  and  challenges  for  a  Systems 
Engineering  Transformation 

o  "Future  -  How"  -  Blending  and  evolving  our  research  results  with  Digital  Engineering 
(DE)  Transformations  across  the  DoD  to  be  in  a  Future  State  by  Computationally 
Enabled  DE 

o  Deep  Dive  a  Few  Research  Topics 

o  Integrated  Systems  Engineering  Decision  Management  (ISEDM)  Process  Enabled  by 
Digital  Engineering  Technologies,  presented  by  Dr.  Matthew  Cilli 
o  Semantic  Technologies  and  Ontologies  Research  to  enable  Trade  Space  Analytics  for 
Engineered  Resilient  Systems,  presented  by  Dr.  George  Ball 
o  Breakout  Session  discussing 

•  Risk  for  Digital  Engineering  Transformation 

•  Priorities  for  Digital  Engineering  Transformation 
o  Forward  Planning  and  Actions 

■  Working  session  #5:  l-August-2017  held  at  Stevens 

o  Perspectives  on  July  31  Session:  Systems  Engineering  Transformation  through  Model 
Centric  Engineering 
o  ARDEC  challenge  updates 

o  Presentation  and  demonstrations  on  Integration  and  Interoperability  Framework 
(lolF)  overview  and  demonstration  (UC09,  UC00,  UC01,  UC02,  UC04),  and  lolF  model 
and  workflow  representation 

o  Overview  of  OpenMBEE  plan  for  integration  into  the  lolF 
o  Decision  Framework  (UC06)  and  Formalizing  Assessment  Flow  Diagram  through 
MDAO  (UC03) 

o  Status  updates  of  the  Graphical  CONOPS  (UC01)  integration  with  MDAO  (UC03) 
o  Status  update  from  UCE/ICE 
o  Next  steps  for  Phase  II 


2.4  Tentative  Schedule  for  Meeting,  Demonstrations  and  Deliverables 

Table  1  provides  a  list  of  the  deliveries,  demonstrations  and  discussions  for  our  bi-weekly  status 
and  other  meetings  involving  our  ARDEC  sponsors. 
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Table  1.  Schedule  for  Demonstration  and  Deliverables 


Date 

Demo  /  Presentation  /  Reports 

Status 

Sep  21  &  22, 
2016 

1st  Working  Session  at  ARDEC  -  see  meeting  notes. 

Done 

Nov  4,  2016 
(Fri) 

Mission  Level  Modeling  and  Graphical  CONOPS  (2 
approaches) 

•  Paul  Grogan 

•  Roger  Blake 

•  Roger  Jones 

Done 

Nov  7,  2016 

Interim  Report/Bi-Monthly  Status 

•  Expand  on  all  tasks  that  are  mapped  to  Use  Cases 
project  model 

Done 

Nov  22,  2016 

Decision  Framework  Approach  by  Matt  Cilli / Robin  Dillon 

Done 

Dec  2,  2016 

MDOA  presentation  and  demonstration  by  Steven 
Hoffenson 

Discussion  of  Mission/System  Simulations  Roger  Jones, 
Roger  Blake,  Paul  Grogan 

Done 

Dec  16,  2016 

Design  of  a  Systems  Representation  Framework  for 
Counter  UAS  Operations  by  Kishore  Pochiraju 

Done 

Dec  20,  2016 

Information  Model/Ontology  by  Mark  Blackburn  /  Mary 
Bone  /  Gregg  Vesonder 

Done 

Jan  10,  2017 

2nd  Working  Session  at  ARDEC  -  see  meeting  notes. 

Done 

Jan  15,  2017 

Update  Interim  Report/Bi-Monthly  Status 

•  Expand  on  tasks  that  are  mapped  to  use  cases  in 
project  model 

Done: 

This  report 

Jan  25-27, 

2017 

NASA/JPL  Symposium  and  Workshop  on  Model  Based 
Systems  Engineering 

Meeting  notes 
delivered 

Jan  28-31, 

2017 

INCOSE  International  Workshop 

Meeting  notes 
delivered 

Feb  10,  2017 

Demonstrations  of  Graphical  CONOPS 

■  Roger  Jones  -  Unity  gaming  of  competing 
autonomous  quadcopters 

■  Todd  Richmond  -  Video  of  Unity  gaming  for  Early 
Synthetic  Prototyping 

Done 

Feb  24,  2017 

Automatic  Concurrent  Engineering  and  Knowledge-Based 
Product  Design  and  Manufacturing  (Kishore  Pochiraju) 

Done 

Mar  2,  2017 

Semantic  Web  Technologies  (Mary  Bone  /  Mark 
Blackburn) 

Done 

Mar  7,  2017 

Syndeia  Demonstration  (Manas  Majaj  /  Jeff  McDonald) 

Done 

Mar  9,  2017 

ARDEC  sponsor  Eddie  Bauer  participated  in  NAVAIR,  RT- 
170  working  session  #29  at  NAVAIR. 

Done 

Mar  10,  2017 

Update  on  HLA  approach  (Roger  Blake  /  Paul  Grogan) 

Done 
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Mar  15,  2017 

Update  Interim  Report 

Expand  on  tasks  that  are  mapped  to  use  cases  in  project 
model 

Done: 

Prior  version 
of  this  report 

Mar  24,  2017 

Mary  Bone  gave  a  talk  on  ontologies  as  it  related  to 
AAMODAT  and  Challenge  area  #5 

Done 

Mar  30,  2017 

Working  Session  #3  at  Stevens  -  see  meeting  notes. 

There  were  over  25  attendees,  including  nine  (9)  from 
ARDEC 

Done 

Apr  7,  2017 

Kishore  gave  a  talk  on  Design  Automation 

Done 

Apr  18,  2017 

Two  related  talks  on  OpenMBEE  model  in  SysML  to 
support  analysis  of  requirements  development/review  for 
AVCE  iMBE  (Mark  Blackburn) 

Done 

Apr  21,  2017 

Broader  aspects  of  OpenMBEE  (Mark  Blackburn) 

Done 

May  15,  2017 

Bi-monthly  status  report 

•  Expand  on  tasks  that  are  mapped  to  use  cases  in 
project  model 

Done 

May  19,  2017 

Model  Centric  Engineering  Architecture  (Roger  Blake  / 
Paul  Grogan) 

Done 

Jun  2,  2017 

Overview  on  Model  Development  Kit  (MDK)  DocGen  View 
and  Viewpoints  that  were  added  to  AVCE  requirements 
model  to  illustrate  the  DocGen  capabilities  (Benjamin 
Kruse) 

Done 

Jun  13,  2017 

Working  Session  #4  at  ARDEC  -  see  meeting  notes. 

Done 

Jun  30,  2017 

Two  talks  on  Model  Centric  Engineering  Architecture  and 
the  Prototype  of  the  Integration  and  Interoperability 
Framework  (lolF)  and  demonstration  interoperability  using 
semantic  web  technologies  and  ontologies  (Paul  Grogan, 
Roger  Blake,  Mary  Bone,  Chris  Synder,  Harsh  Kevadia) 

Done 

Jul  14,  2017 

Decision  Framework  update  with  discussion  of  use  of 
semantic  web  technologies  and  concept  for  modeling  the 
Assessment  Flow  Diagram  (Matt  Cilli,  Robin  Dillon-Merrill, 
Mary  Bone,  John  Dzielski) 

Done 

Jul  15,  2017 

Updated  Interim  Report 

•  Expand  on  tasks  that  are  mapped  to  use  cases  in 
project  model 

Done 

July  31,  2017 

Systems  Engineering  Transformation  through  Model 
Centric  Engineering  Past,  Present,  and  Future  -  Special 
Session  at  Stevens  (Mark  Blackburn,  Dinesh  Verma) 

Done 

Aug  1,  2017 

Working  Session  #5  at  Stevens 

Done 

Aug  8,  2017 

Final  Technical  Report 

Done: 

This  report 
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Part  II:  Task  Detail  Summary 


The  material  in  Part  II  provides  additional  detail  on  the  latest  status  on  the  tasks  in  the  context 
of  the  research  use  cases,  including  information  shared  during  some  of  the  working  sessions 
and  bi-weekly  meetings.  An  extensive  amount  of  material  covered  in  Part  II  of  the  RT-141  final 
report  [22]  and  RT-157  final  report  [23]  still  provides  relevant  information  to  this  research,  but 
has  not  been  integrated  into  this  report. 

Each  of  these  sections  has  a  team  of  researchers,  which  are  reflected  by  Figure  3.  We  are 
adding  the  information  from  the  different  perspectives,  and  will  continue  to  integrate  the  story 
as  the  research  results  evolves  through  Phase  II  (August  2017  -  August  2018). 

3  Information  Model  (UC00) 


MCE  is  enabled  by  computational  technologies  that  now  provide  a  means  for  using  modeling 
and  simulation  in  a  transformed  approach  to  systems  engineering.  A  key  problem  is  that  most 
of  these  technologies  are  not  integrated  (and  many  may  never  be).  Therefore,  we  are 
interested  in  an  approach  to  using  data  interoperability  as  a  means  (or  surrogate)  for 
accomplishing  integration.  This  is  challenge  area  #3,  which  has  now  been  extended  to 
incorporate  this  concept  under  challenge  area  #5,  and  defined  in  more  detail  under  UC10  (see 
Section  13). 

This  information  model  characterizes  the  underlying  information  and  relationships  to 
"everything"  that  might  need  to  be  produced  by  the  tools  of  AVCE.  We  are  using  OWL  and  SWT 
to  represent  the  information.  Our  efforts  with  ARDEC  are  also  complemented  by  our  efforts 
with  NAVAIR  and  the  Semantic  Technologies  for  Systems  Engineering  initiative  (ST4SE)  that  was 
established  in  April  2017. 


3.1  Semantic  Technologies  for  Systems  Engineering 

Briefly,  the  SWTs  are  based  on  a  standard  suite  of  languages,  models,  and  tools  that  are  suited 
to  knowledge  representation.  Figure  5  provides  a  perspective  on  the  SWT  stack,  which  includes 
extended  Markup  Language  (XML)  [129],  Resource  Description  Framework  (RDF)  [180]  and 
Schema  (RDFS),  Web  Ontology  Language  (OWL)  [179]  (i.e.,  OWL2),  the  SPARQL  Protocol  And 
RDF  Query  Language  (SPARQL)  [181],  and  others.  RDF  can  describe  instances  of  ontologies  - 
that  is,  the  data  for  particular  model  instances,  where  OWL  relates  more  to  metamodels 
describing  the  class  of  information  that  can  be  characterized  as  RDF  instances.  RDFS  extends 
RDF  and  provides  primitives  such  as  Class,  subClassOf,  and  subPropertyOf.  The  SWT  was 
created  to  extend  the  current  Internet  allowing  combinations  of  metadata,  structure,  and 
various  technologies  enabling  machines  to  derive  meaning  from  information,  both  assisting  and 
reducing  human  intervention.  This  technology  is  generally  applicable  to  many  different 
applications,  and  our  research  is  beginning  to  reflect  that  from  the  demonstrations  of  the  lolF, 
to  the  Decision  Framework,  and  communicating  the  uses  of  SWT  by  NASA/JPL,  and  how  such 
capabilities  can  be  integrated  within  a  model  based  engineering  environment,  like  OpenMBEE 
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to  provide  additional  reasoning  on  the  information  that  is  captured  such  as  completeness, 
consistency  and  well-formedness. 


Layers  of  Abstraction 


Ontology  and  reasoning 
layers 

Analytical  Knowledge 


Data  layers 


predicate 


RDF  triple 


Representation  /  syntax  layers 


Semantic  Web  Technology  Stack 


Applications  and  Interfaces 


Trust 


Proof 


Unifying  Logic 


SPARQL 


Ontology:OWL 


RDFS 


Rules: 

RIF 


Data  lnterchange:RDF 


XML 


Unicode 


URI 


Figure  5.  Semantic  Web  Technologies  related  to  Layers  of  Abstraction 

Figure  6  provides  another  perspective  using  an  instantiation  created  by  NASA/JPL,  which 
reflects  a  number  of  the  pieces  we  are  interested  in  using: 


■  Three  core  elements  of  View  Editor,  DocGen  and  Model  Management  System  (MMS) 

■  MagicDraw  client  (in  which  the  MDK/DocGen)  plugin  works 

■  Teamwork  Cloud  server  from  NoMagic  is  used  with  MMS 

■  The  NASA  ontologies  for  Systems  Engineering  used  to  check  constraints  (e.g., 
consistency,  completeness,  well-formedness)  [90]  related  to  the  model  is  shown  in 
Figure  7 

o  These  are  being  open-sourced 

o  We  would  like  to  opportunistically  leverage  these  capabilities  both  with  NAVAIR  and 
ARDEC  through  our  efforts  with  the  ST4SE 
o  These  ontologies  have  grown  out  of  a  history  of  work,  including  the  INCOSE 
modeling  patterns  group 
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Multidisciplinary  Design, 
Analysis,  and  Optimization 
(MDAO)  platform 
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SE  Modeling 
Patterns  formalized  as 
Ontologies 

Semantic  Web  Technologies 
support  Continuous  Checks 
and  Model  Measures 

View  Editor 


DocGen 


Visualization 


Model  Management 
System  (MMS) 
Single  Source  of 
Truth  (SST) 


*An  Integrated  Model  Centric  Engineering  (IMCE)  Reference  Architecture  for  a 
Model  Based  Engineering  Environment(MBEE),  NASA/JPL,Sept,2014. 


Figure  6.  NASA/JPL  Instantiation  of  OpenMBEE  (circa  2014) 

The  following  figures  have  been  taking  from  Model-Centric  Engineering,  Part  3:  Foundational 
Concepts  for  Building  System  Models  [91].  Figure  8  shows  the  Integrated  Model  Centric 
Engineering  (IMCE)  concept  that  is  being  developed.  The  process  involves: 

■  Creating  ontologies  for  foundational  systems  engineering  derived  from  the  modeling 
patterns  (reflected  in  Figure  7) 

o  This  can  be  done  in  any  OWL  modeling  tool  such  as  the  open  source  Protege 
o  The  ontologies  are  turned  into  SysML  profiles 

o  The  SysML  profiles  are  loaded  into  a  modeling  tool  for  creating  models 
o  The  profiled  SysML  models  are  exported  back  into  OWL  statements 
o  Checks  for  completeness,  consistency  and  well-formedness  can  performed 
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@r  Partial  Map  of  Foundation  Ontology  Concepts 
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Figure  7.  NASA/JPL  Foundational  Ontology  for  Systems  Engineering 
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IMCE  Vision  for  OWL/SysML  Integration 


Model-Based  Systems  Engineering 
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This  is  one  example  of 
how  OWL  and  SysML  tools 
might  be  used  in  MBSE 

_ 7 

Figure  8.  From  Ontologies  to  SysML  Profiles  and  Back  to  Analyzable  OWL  /  RDF 

Figure  9  shows  the  various  representations  associated  with  the  concept  described  in  Figure  8: 

1.  The  modeled  statement  in  English  is:  "Component  performs  Function" 

2.  The  OWL/RDF  representation  of  the  statement  in  low-level  XML  for  this  same  statement 

3.  The  Profile  and  Stereotypes  used  in  the  model  (loaded  into  a  SysML  model) 

4.  The  Stereotypes  used  in  a  SysML  Block  Definition  Diagram  (BDD) 
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/Sk/  (animated) 

English  *  OWL  SysML  Profile  Usage 


Model-Based  Systems  Engineering 


|  ^  j  English:  “Component  performs  Function” 


OWL  (RDF) 
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Figure  9.  Multiple  Representations  in  Process 


We  believe  that  SWT  has  the  potential  to  contribute  significantly  to  challenge  #3  and  challenge 
#5.  As  reflected  in  Figure  1,  the  challenge  is  to  characterize  this  information  for  each  of  the 
various  domains,  including  requirements,  risks,  designs  (e.g.,  electrical,  mechanical,  etc.),  and 
analyses.  The  use  case  diagram  in  Figure  3  reflects  why  there  are  so  many  associations  from  the 
other  use  cases.  In  addition,  we  believe  that  it  is  technically  feasible  to  capture  this  information 
and  provide  it  as  input  to  the  Decision  Framework  and  AAMODAT  tool.  The  research  approach 
is  to  use  SWT  to  demonstrate  the  concept  to  both  characterize  the  data  and  information  as  well 
as  rules,  and  query  language  for  processing  and  data  exchange.  Several  briefings  on  SWT 
concepts  (e.g.,  ontologies)  and  example  uses  have  been  provided  in  both  working  session  and 
webinar  sessions.  Challenge  area  #5  has  been  defined  and  the  prioritization  of  the  information 
model  will  align  with  the  objective  to  characterize  the  information  and  rules  associated  with 
inputs  to  AAMODAT;  as  such,  this  is  now  defined  as  a  new  user  case  UC10. 


The  third  and  fifth  working  session  demonstrated  showed  evolving  capabilities  to  illustrate 
broader  viability  of  the  concept  of  using  data  interoperability  defined  by  ontologies  as  a  means 
for  collecting,  managing  and  composing  information  collected  across  domains.  This  is  a 
strategic  thrust  area  moving  into  Phase  II. 


We  are  evolving  an  Integrating  and  Interoperability  Framework  (lolF)  as  part  of  UC09  as  shown 
in  Figure  4.  We  are  working  with  other  use  case  teams  to  provide  a  demonstration  of  Decision 
Framework  enabled  by  semantic  technology  (UCOO).  We  envision  SWT  and  Decision  Layer 
(UC10)  highlighted  in  orange  oval  to  be  in  this  part  of  the  concept.  In  collaboration  with  NAVAIR 
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and  NASA/JPL,  we  would  also  like  to  bring  in  the  IMCE  ontologies  for  systems  engineering.  We 
are  considering  using  tool-to-tool  integration  as  discussed  in  UC09,  Data  Acquisition  and 
Aggregation  in  research  to  integrate  Graphical  CONOPS  (UC01),  and  Mission  and  System 
Operational  Capabilities  (UC02). 


3.2  UCOO  Mapping  to  Other  Use  Cases 

We  plan  to  continue  research  in  the  other  two  areas  of  Graphical  CONOPS  (UC01)  and  Mission 
and  Systems  (UC02)  to  understand  the  flow  of  information  needed  to  be  linked  between  them, 
and  characterize  those  linkages  in  an  Information  Model.  The  information  produced  under  the 
following  use  cases  has  begun  to  characterize  elements  of  the  metamodels,  for  example: 

■  Parameters  in  the  Graphical  CONOPS 

■  High  Level  Architecture  (HLA)  metamodel  for  both  VT  MAK  [103]  and  Distributed 
Simulation 

■  Early  Synthetic  Prototyping  (ESP)  Data  Structures  and  Reasoning 

■  Lack  of  "acceptable"  representations  and  transformation  using  SysML 
o  Graphical  diagrams  specified  at  multiple  abstractions 

o  Oriented  towards  concrete  design  (=very  detailed) 
o  Likely  to  be  missing  relevant  mission/scenario  parameters 
o  XMI  difficult  to  'query'  for  structural  parameters 
o  Low-level  with  extensive  unique  IDs  difficult  to  interpret/parse 
o  Behavioral  diagrams  cannot  easily  be  transformed  to  scripted  code  (e.g.  Lua  script) 

As  discussed  by  the  ARDEC  challenge  area  #1  team,  which  relate  to  UC03,  UC04,  and  UC05 
involve  the  need  to  improve  the  integration  of  architectural,  system  and  component  models 
across  the  domains,  and  better  link  with  other  modeling  and  simulation  capabilities  targeted  to 
specific  disciplines.  At  the  system  level  they  may  be  developed  using  MBSE  methods  and  be 
represented  in  standard  modeling  languages  such  as  SysML  [131].  The  linkages  between  the 
MBSE  and  design  disciplines,  usually  referred  to  as  Model-Based  Engineering  (MBE),  is  often  not 
precisely  represented,  with  a  few  exceptions.  When  it  is  done  using  tool-to-tool  integration, 
such  integrations  can  be  rather  susceptible  to  tools  updates  [36].  We  believe  there  are 
opportunities  to  address  this  need  in  more  tool  agnostic  ways  using  SWT.  See  UC09  and  UC10. 

A  key  reason  for  the  need  for  cross-domain  model  integration  is  the  underlying  complexity 
needed  to  accomplish  the  scenarios  associated  with  Figure  10.  In  addition,  our  research  as 
illustrated  by  the  DARPA  META  project  [8]  has  shown  that  methods  are  needed  to  ensure  that 
the  tools  provide  the  expected  automation,  efficiencies,  and  produce  the  desired  information. 
This  points  to  the  need  for  both  methods  (Task  3),  and  because  many  of  the  modeling  and 
simulation  capabilities  that  may  be  integrated  into  an  MDAO  workflow  can  be  modeling  and 
simulation  capabilities,  they  require  some  type  of  assessment  to  ensure  the  integrity  of  the 
predictions. 
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Component  Models 
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Detailed  Behaviors 


Iterative  Process 


Figure  10.  Integrate  Multiple  Levels  of  System  Models  with  Discipline-Specific  Designs 

We  believe  there  are  research  challenges  to  better  quantify  design  margins,  parameter 
uncertainties,  and  system  performance  sensitivities  associated  with  physics-based  digital 
models.  There  are  opportunities  and  challenges  in  integration  of  relevant  multi-physics 
modeling  and  simulation,  need  for  earlier  high-fidelity  models,  and  means  to  assess  reduced- 
order  models.  In  addition,  there  are  needs  for  determining  optimal  risk/cost  tradeoff  for 
continual  Verification,  Validation  and  Accreditation  (VV&A)  or  alternative  means  for  assessing 
trust  in  model  and  simulation  predictions. 

As  shown  in  Figure  11  [50],  there  can  be  a  very  large  set  of  tools  that  can  be  used  to  develop 
the  needed  data  and  information  across  all  of  the  domains.  Therefore,  it  is  important  that 
appropriate  methods  are  applied  to  the  selected  tools  that  are  assembled  for  use  on  a  project 
or  program.  As  a  secondary  objective  that  is  being  demonstrated  as  leading  edge  approach  by 
NASA/JPL  is  to  ensure  models  are  created  that  comply  with  established  modeling  patterns.  We 
provide  information  at  the  second  working  session  on  the  NASA/JPL  approach,  which 
transforms  the  model  information  into  a  tool-neutral  SST  based  on  ontologies,  and  then  uses 
standard  SWT  to  apply  checks  to  ensure  completeness  and  consistency  [90]. 
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Figure  11.  Appropriate  Methods  Needed  Across  Domains 


3.3  UCOO,  UC07  and  UC10  (Decision  Framework  and  AAMODAT) 

As  shown  in  Figure  3,  this  task  relates  to  UC07  and  UC10.  We  have  confirmed  with  Dr.  Matt  Cilli 
that  we  believe  the  information  captured  can  be  used  to  drive  the  Decision  Support  Model 
Construct  [41]  (referred  to  as  Decision  Framework)  as  shown  in  Figure  2.  We  believe  that  this 
concept  and  process  has  been  demonstrated  to  provide  senior  management  and  program 
managers  with  visual  representation  of  key  tradeoff  defined  in  terms  of  Performance,  Cost, 
Time  and  Risk. 

The  key  concept  associated  with  the  information  model  and  the  decision  framework  is  to  work 
with  Matt  Cilli,  Cliff  Marini  (developer  of  AAMODAT)  and  Robin  Dillion-Merrill  on  exploring 
potential  enhancements  and  extensions  to  the  Integrated  Systems  Engineering  Decision 
Management  (ISEDM)  process  and  the  related  decision  support  tool  AAMODAT.  Some  of  the 
objectives  for  the  new  challenge  area  #5,  focused  on  how  to  integrate  cross  domain  models 
(SysML  model,  Engineering  Models,  Performance  Models,  Cost  Models,  etc.)  with  decision 
support  model  (AAMODAT)  while  executing  ISEDM  process.  This  is  specifically  where  ARDEC 
requested  us  to  demonstrate  ability  to  create  Domain  Ontology  via  AAMODAT  views,  which  is 
discussed  in  more  detail  in  UC10  (see  Section  13). 

4  Graphical  CONOPS  (UC01) 


There  are  nine  different  modeling  and  simulation  examples  that  are  being  developed  to  support 
UAS  and  Counter  UAS  analysis  case  study.  These  different  approaches  involve  different 
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researchers,  and  look  at  the  problems  using  different  technologies,  both  in  terms  of  types  of 
abstractions,  level  of  fidelity,  no  human-in-the-loop,  and  humans-in-the-loop,  which  also  have 
an  impact  on  trading  off  cost  and  value  of  the  simulation.  Each  approach  is  described  in  the 
subsection  below.  Fundamentally,  we  are  also  interested  in  the  information  (metamodels, 
which  map  to  OWL)  and  associated  methods  to  produce  and  analyze  this  information  in  order 
to  integrate  with  the  other  models  in  use  cases  UC01,  UC02,  UC03,  UC04,  and  UC05. 


4.1  UAV  CONOPS  using  Gaming  Engine  Simulation  (Jones  View) 

Engineered  systems  have  advanced  to  the  stage  in  which  they  share  many  properties  with 
biological  and  sociological  systems.  Engineered  systems  can  have  systems  embedded  in  them, 
and  those  subsystems  can  have  subsystems  embedded  in  the  subsystems.  This  is  reminiscent  of 
the  layered  level  of  complexity  in  biology.  Molecular  processes  form  cells;  cells  form  organs; 
organs  form  organisms;  and  organisms  form  societies.  In  some  cases,  engineered  systems  are  a 
part  of  sociological  systems.  A  city  is  a  combination  of  a  social  system  and  many  engineered 
systems,  from  traffic  systems  to  the  power  grid. 

Nature  has  solved  many  of  the  problems  that  systems  engineers  are  struggling  with.  These 
problems  include  incompatibility  of  systems,  multidisciplinary  integration,  incompatible  time 
scales,  systems  of  systems,  and  more.  Can  we  examine  the  manner  in  which  Nature  solves 
many  of  these  problems  to  inform  the  design  and  optimization  of  complex  engineered  systems? 
This  use  case  addresses  at  least  this  question. 

Biological  and  sociological  systems  are  not  designed  in  the  traditional  sense.  The  designs 
emerge  from  interaction  with  each  other  and  with  the  system  environment  through  a  process 
of  evolution  and  natural  selection. 

The  goal  of  this  research  is  to  identify  a  general  systems  framework  that  can  be  used  as  a 
backend  for  Graphical  CONOPS  in  support  of  MDAO  as  well  as  provide  inputs  to  other  types  of 
modeling  and  simulation,  such  as  both  2D  and  3D  approaches  to  mission  and  system 
simulation.  Since  Nature  has  solved  many  of  the  systems  problems,  the  framework  will  be 
organically-based.  The  framework  will  be  able  to  create  models  of  a  very  large  class  of  systems 
and  systems-of-systems.  As  shown  in 

Figure  12,  we  have  created  an  example  that  has  demonstrated  the  use  of  this  concept  in  an 
environment  involving  UAS  mission  scenarios  using  the  Unity  Gaming  Engine;  this  will  be  the 
canonical  example. 

Roger  Jones  has  demonstrated  a  Graphical  CONOPS  created  using  the  Unity  game  engine  that 
that  provides  Monte  Carlo  simulation  feedback  to  MDAO.  There  are  two  possible  surveillance 
missions  for  a  blue  quadcopter.  In  scenario  one,  the  blue  quadcopter  searches  for  an  object, 
and  mission  is  unimpeded.  In  the  second  mission,  a  red  quadcopter  actively  tries  to  prevent  the 
blue  copter  from  succeeding  at  its  mission,  as  shown  in  Figure  12.  Both  quadcopers  are  fully 
autonomous.  There  are  options  to  change  different  parameters  related  to  the  two  UAVs  in  a 
dynamic  manner.  As  shown  in 
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Figure  13,  there  are  also  tabs  that  can  be  used  to  parametrically  modify  the  capabilities  of  the 
two  different  UAVs. 

■  The  latest  version  (1.09)  provides  to  feature: 

o  Communication  with  other  software  through  JSON  files 

o  Plan  to  use  MDAO  is  performed  in  Excel,  which  writes  to  JSON  that  is  read  by  the 
Unity  gaming  engine 

o  Has  more  realistic  battery  and  flight  models 

o  Enhanced  design  interface  that  allows  user  to  quickly  explore  design  space  around 
an  optimum  determined  by  static  MDAO  software 

■  The  planned  next  steps  include: 

o  Complete  analysis  and  optimization  modules  and  integration  with  ModelCenter  or 
other  simulations  through  JSON  files 

o  Integrate  a  synchronized  simulation  with  the  output  from  the  graphical  CONOPS 
being  published  through  the  SWT  and  be  consumed  (subscribed)  through  the  SWT 
by  the  2D  simulation 

o  Demonstrate  integrated  simulation  as  part  of  the  lolF 


Figure  12.  Unity  Gaming  Engine  Simulation  of  Two  Moving  UAV  with  Camera 
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Figure  13.  Unity  Gaming  Engineering  Simulation  MDAO 


4.2  Graphical  CONOPS  (USC ICT  -  Richmond  View) 

The  USC  Institute  for  Creative  Technologies  (ICT)  will  support  this  use  case  by  investigating 
various  aspects  of  Early  Synthetic  Prototyping  (ESP)  capability  that  has  been  developed  for 
RDECOM-ARDEC.  They  too  use  the  Unity  gaming  platform  with  other  technologies  that 
integrate  and  study  humans-in-the-loop.  The  scope  of  work  includes,  but  is  not  limited  to: 

■  Visualization  of  tradespace  and  alternatives 

■  Graphical  CONOPS  improvements 

■  Assess  collaboration  opportunity  with  TRADOC's  ESP 

■  Provide  recommendations  for  Collaborative  Design  Infrastructure 

The  ESP  project  explores  options  to  leverage  emerging  synthetic  immersive  environments  to 
foster  innovative  design  and  testing  as  shown  in  Figure  14.  ESP  seeks  to  bring  the  Soldier  (i.e. 
the  end  user)  into  the  design  and  testing  process  during  initial  planning  stages,  helping  to 
connect  those  that  design/build  (engineers)  and  those  that  employ  (Soldiers).  ESP  also  is  being 
designed  to  enable  testing  of  nascent  concepts  and  explore  not  only  the  art  of  the  possible  for 
today,  but  also  the  innovations  of  tomorrow.  The  concepts  of  early  fidelity  and  minimum  viable 
model  are  critical  for  speed  and  agility  and  the  ideation  design  and  testing  process.  There  are 
some  videos  available  to  illustrate  more  aspects  of  the  ESP  capabilities: 

■  https://vimeo.com/145230112 

■  https://vimeo.com/139283830 

■  https://vimeo.com/139283668 
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Figure  14.  Early  Synthetic  Prototyping  Video  Example 


The  proliferation  and  maturation  of  tools  supporting  virtual  environments  combined  with 
emerging  immersive  capabilities  (e.g.  Oculus  Rift  and  other  head  mounted  displays)  point 
towards  the  ability  to  take  nascent  ideas  and  realize  them  in  engaging  ways  through  a 
virtual/synthetic  prototyping  system.  In  effect,  "bend  electrons  before  bending  metal," 
enabling  Soldier  (end-user)  feedback  early  in  the  design  process,  while  fostering  an  atmosphere 
of  collaboration  and  innovation.  Simulation  has  been  used  in  a  variety  of  ways  for  concept, 
design,  and  testing,  but  current  methods  do  not  put  the  user  into  the  system  in  ways  that 
provide  deep  feedback  and  enable  a  dialogue  between  Warfighter  and  Engineer  (as  well  as 
other  stakeholders)  that  can  inform  design,  and  more  broadly,  the  entire  acquisition  process. 
The  key  is  to  fail  early  when  it  is  cheap,  rather  than  late  in  the  process. 

ESP  is  different  from  existing  game/simulation  engines.  Current  synthetic  environments  track 
fairly  traditional  metrics  giving  data  largely  as  scores  around  easily  quantifiable  outcomes.  At 
the  core  of  ESP  moves  towards  a  new  generation  of  metrics  and  analytics  that  focus  on  the 
wants  and  needs  of  the  user,  tracking  not  only  their  in-game  performance  -  what  they  did  -  but 
also  their  inner  motivations  how  and  why  they  did  things  and  how  they  feel  at  specific  points  in 
time  during  the  interaction.  In  order  to  provide  useful  and  untapped  information  back  to  a 
designer/engineer,  ESP  will  need  to  assess  a  number  of  softer  metrics  such  as  user  frustration. 
In  addition,  deeper  granularity  will  be  tracked  as  well  as  challenges  such  as:  sources  of 
frustration  for  stakeholders. 

ESP  is  currently  in  the  early  prototype  stage,  and  in  fact,  is  creating  ESP  by  using  an  evolving  ESP 
conceptual  model,  understanding  the  requirements  that  enable  creativity  and  innovation 
through  virtual  engagement.  These  exploratory  environments  are  multi-player  and  are 
exploring  the  design  of  next-generation  vehicles  as  well  as  their  use  in  a  variety  of  contexts. 
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Users  can  make  modifications  on-the-fly,  and  help  find  new  ways  to  not  only  design  and  build, 
but  also  employ  the  systems.  Play  can  enable  emergent  behaviors  to  arise  and  be  tracked, 
teased  out,  and  assessed.  The  broader  ESP  effort  also  includes  research  work  at  Naval 
Postgraduate  School  (NPS),  collaboration  with  various  Army  Research  Development  and 
Engineering  Center  (RDECs),  and  other  Army  partners.  The  initial  prototypes  are  undergoing 
iteration  and  testing  and  will  inform  the  ESP  design  and  requirements,  as  well  as  facilitate 
ongoing  research  into  four  vectors:  analytics,  idea  ingest,  emerging  interfaces,  and  community. 

The  vision  for  ESP  is  an  integrated  ecosystem  where  community  stakeholders  may  propose, 
develop,  test,  discuss  and  refine  ideas,  capabilities,  and  concepts  of  operation  within  a  virtual 
prototyping  environment.  These  can  connect  to  ideation  platforms  (on  the  left),  and  to  higher 
fidelity  modeling  and  simulation  (on  the  right). 

Edgar  Evangelista  showed  a  video  at  the  third  working  session  on  the  work  they  were  doing  at 
USC-ICT,  bringing  soldiers  early  in  the  design.  They  are  developing  army  games  to  deploy  with 
soldiers  in  a  multi-player  environment.  The  key  point  is  getting  early  prototype  systems  into  the 
hands  of  the  soldiers.  The  virtual  world  allows  soldiers  to  play  around  with  new  concepts  that 
we  don't  want  to  build  in  the  physical  world. 


4.2.1  Tradespace  for  Early  Synthetic  Prototyping  (ESP) 

Previous  Army  Capabilities  Integration  Center  (ARCIC)-funded  efforts  resulted  in  a  series  of 
prototype  applications  centered  around  the  concept  of  Early  Synthetic  Prototyping.  ICT  created 
multi-player  game  (Unity3D  engine)  with  a  red  vs.  blue  "capture  the  flag"  game  mechanic.  The 
environment  allowed  players  to  choose  and  modify  vehicles,  and  a  wide  variety  of  data  was 
captured  during  play,  including  biometrics  (https://vimeo.eom/1 351 00689).  In  order  to 
visualize  the  data,  a  Post  Exercise  Analysis  (PEA)  application  was  developed  that  ingested  game 
logs  and  provided  interactive  replay  and  analysis  (https://vimeo.eom/1 39283668).  The  results 
of  these  prototype  applications  led  to  current  work  by  Army  Game  Studio  (AGS)  to  develop  the 
next  iterations  of  ESP. 

ICT  has  been  working  to  identify  and  analyze  the  gameplay  data  from  ESP  systems.  ICT  is 
currently  working  with  the  AGS  to  collect  player  gameplay  for  the  ESP  system.  There  are 
tradeoffs  between  different  data  storage  formats,  as  well  as  effort  needed  to  translate  between 
formats  for  post-game  analysis.  ICT's  current  research  has  led  them  to  store  all  the  data  of  the 
ESP  multiplayer  system  so  that  they  may  analyze  the  data  post-game.  They  currently  allow  for 
replay  of  the  system.  However,  they  do  need  to  investigate  how  to  organize  the  data  to  direct  it 
toward  experiment  objectives.  Currently,  they  are  relying  on  AGS's  Operation  Overmatch  to 
bring  in  enough  participants  and  gameplay  recordings,  so  that  they  can  help  researchers 
develop  and  run  experiments  on  the  efficacy  of  certain  systems.  ICT  has  been  working  with  AGS 
to  identify  methods  of  storing  thorough  gameplay  data  for  analysis. 

There  are  differences  between  the  AGS's  data  storage  format  and  ICT's  data  storage  format 
which  poses  certain  translation  issues  as  well  as  introducing  interesting  tradeoffs.  Currently, 
AGS's  data  storage  format  relies  on  a  fixed  logging  rate  with  an  implicit  format  versus  ICT's 
variable  log  rate  with  an  explicit  format.  Their  fixed  logging  rate  stores  the  position  of  each  unit 
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every  timestep,  while  not  actually  recording  the  timestamp.  The  timestamp  is  implicit.  This  also 
implies  that  AGS  will  store  events  within  the  order  they  occur.  The  data  format  designer  is 
required  to  be  consistent  in  implying  event  (e.g.,  firing,  damage)  ownership  of  the  particular 
unit  so  that  certain  events  can  be  tracked  and  replicated.  This  requires  thorough 
documentation  for  other  parties  to  ingest  the  data.  Further,  in  the  case  that  a  unit  stays  still, 
this  adds  redundant  data  for  its  position  since  it  is  stored  every  timestep.  This  method  is  sound 
as  long  as  other  developers  who'd  like  to  use  the  data  are  well  informed  of  the  implied  data. 

Conversely,  ICT's  PEA  data  format  uses  timestamps  for  variable  log  rates  and  explicitly  states 
event  ownership  and  damage  states.  Although  this  requires  documentation  as  well,  the  explicit 
data  format  does  make  it  easier  to  ingest  initially.  However,  it  does  have  a  slightly  larger 
memory  footprint.  Variable  log  rates  also  cover  frame  skipping  issues,  in  which  the  simulation 
computer  may  not  be  able  to  log  the  status  of  a  certain  frame.  However,  an  issue  with  variable 
log  rates  also  requires  certain  playback  to  account  for  the  variability.  If  certain  physics  events, 
like  projectiles,  are  to  be  replicated,  they  need  to  check  the  timestamp  and  interpolate  the 
position. 

Both  methods  are  found  to  be  sufficient  in  storing  requisite  event  and  status  data  for  gameplay 
replication.  Either  method  requires  analyzers  to  develop  translation  middleware  to  fit  the  data 
format  into  their  analysis  software.  AGS  has  also  developed  an  easy  way  to  deliver  gameplay 
results  to  analyzers.  They  have  set  up  a  protected  website  that  allows  analyzers  to  download 
batches  of  gameplay.  This  gives  ICT,  as  well  as  other  partners,  easy  access  to  the  results  at  any 
time. 

ICT  has  found  that  there  are  other  data  format  considerations  dependent  on  the  type  of 
gameplay  being  recorded.  They  are  also  tasked  with  developing  design  considerations  for  ESP: 
Higher  Echelon  (ESP:HE).  ESP:HE  is  a  turn-based  strategy  game  aimed  at  educating  captains  on 
troop  movement,  capabilities,  and  other  facets  of  directing  larger  units  in  battle.  Because  this  is 
a  turn-based  game  and  not  a  first-person  game,  ICT  is  more  concerned  with  recording  events, 
rather  than  states  at  each  timestep. 

Further,  not  only  should  ICT  be  concerned  in  storing  diegetic  gameplay  events,  as  they  do  in 
ESP:  Small  Unit  and  Operation  Overmatch,  they  must  also  be  concerned  with  storing  non- 
diegetic  gameplay  events.  ESP:HE  is  a  much  more  complex  strategy  game  with  a  more  diverse 
set  of  units  and  commands.  Therefore,  the  interface  is  also  more  complex  in  presenting  users 
with  numerous  capabilities  in  leading  their  command.  This  requires  a  more  substantial  training 
period  for  users  as  well  as  introduces  usability  issues  with  the  interface.  Storing  certain  non- 
diegetic  gameplay  events,  such  as  time  spent  on  a  dialogue  window  to  time  spent  between  unit 
actions,  can  provide  insight  on  player  motivation  and  player  issues.  We  can  infer  that  if  a  player 
spends  a  lot  of  time  going  through  certain  dialogues  without  committing  to  an  action  that  the 
player  is  either  collecting  various  information  before  acting  or  is  struggling  with  the  interface.  It 
is  possible  that  the  success  of  that  action  may  help  to  distinguish  this. 

One  of  the  main  objectives  of  ESP  is  to  analyze  gameplay  to  answer  research  questions 
regarding  military  systems,  equipment,  and  tactics.  ICT  must  identify  key  data  points  that  are 
pertinent  to  certain  stakeholders,  while  also  identifying  issues  that  might  allow  for 
misinterpretations.  For  example,  they  might  want  to  identify  weapon  effectiveness  in  certain 
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scenarios,  so  they  can  capture  the  percentage  of  hits  and  misses  a  weapon  generates  in 
gameplay.  This  would  give  us  the  accuracy  of  the  weapon.  However,  it  is  important  to  identify  if 
accuracy  stems  from  user  control,  modeling  error,  or  the  weapon's  given  ability.  This  may 
require  further  research  to  receive  more  accurate  analysis. 

Further,  game  designers  may  develop  game  mechanics  that  may  incentivize  players  to  play 
towards  creating  gameplay  data  that  pushes  the  research  question.  For  example,  selecting  to 
use  certain  weapons  can  be  rewarded.  However,  certain  game  mechanic  modifications  can 
change  player  motivation  and,  in  turn,  change  the  accuracy  of  the  data.  So,  versions  of  games 
as  well  as  game  types  must  be  stored  with  the  data. 


4.2.2  Biometrics  During  Gameplay 

ICT  has  also  been  researching  methods  for  collecting  biometric  data  that  is  concurrent  with  the 
subjects'  gameplay.  One  issue  that  arises  is  an  inconsistent  synchronization  of  the  biometric 
data  with  gameplay,  especially  considering  the  various  peripherals  which  produce  different 
signals.  Latency  arises  since  they  run  multiple  hardware  peripherals  from  different 
manufacturers  with  different  software.  Synchronizing  the  timesteps  of  the  signals  is  not  always 
possible  due  to  third  party  code  and  different  temporal  formats.  Further,  there  can  be  a  latency 
associated  with  the  signal  causing  certain  effects  associated  with  a  particular  event.  This  may 
require  error  tolerance  and/or  calibration  built  for  custom  synchronization. 

ICT  has  investigated  Lab  Streaming  Layer  (LSL)  which  allows  them  to  synchronize  streaming  data 
from  various  sensors  with  custom  Unity  Engine  simulations.  LSL  allows  them  to  synchronize 
multiple  data  and  time  marker  sources.  It  also  accounts  for  network  and  transmission  latency. 
Although  LSL  allows  for  (near)  real-time  data  access,  they  also  rely  on  post-game  analysis  to 
synchronize  the  signals. 

Furthermore,  there  may  be  an  issue  of  signal  trustworthiness  wherein  the  ground  truth  may 
not  be  represented.  The  player  may  not  be  a  reliable  subject.  Although  they  may  not  be  able  to 
directly  monitor  each  player,  they  may  be  able  to  use  audio  and  gameplay  recordings  to  find  a 
baseline  to  determine  that  the  player  is  actively  participating. 


4.2.3  Distributed  Interactive  Simulation  Interface  with  Graphical  CONOPS 

ICT  is  also  collaborating  with  ARDEC  to  provide  a  Graphical  CONOPs  in  the  Unity  Engine 
application  with  a  Distributed  Interactive  Simulation  (DIS)  [84]  interface  to  explore  federation  of 
large-scale  simulation  (OneSAF).  The  task  involves  translating  OneSAF  DIS  packets  into  the 
creation  of  a  BLUFOR  (i.e.,  blue  force)  and  OPFOR  (i.e.,  opposing  force)  entities  into  a  given 
environment.  A  BLUFOR  player  will  interact  with  ARDEC's  Gunner  Protection  Kit  and  ARDEC's 
Virtual  Testbed,  which  ICT  will  ingest  into  their  visualization,  so  the  player  can  interact  with  the 
OneSAF  entities.  This  research  is  interested  in  the  ability  to  ingest  input  from  other  simulations 
to  drive  our  visualization  and  ICT  will  document  the  development  of  this  interface  as  well  as  the 
networking  protocol.  The  lolF  capability  may  provide  a  new  approach  to  accomplish  this  need. 
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4.2.4  Mixed  Reality  Prototyping/Collaborative  Design 

ICT,  particularly  the  Mixed  Reality  Lab  (MxR),  continues  to  work  broadly  within  a  construct 
known  as  Mixed  Reality  Prototyping  (MxRP).  This  approach  leverages  the  combination  of  virtual 
environments,  immersive  hardware  and  software  (i.e.,  Augmented  Reality,  Virtual  Reality),  and 
machine  learning  to  enable  design  and  user  testing  to  happen  first  in  synthetic  environments, 
then  in  mixed  reality  spaces  (including  real-time  synchronization  between  virtual  and  physical, 
as  well  as  collaboration  with  distributed  teams 

https  ://www.  youtube. CO  m/watch?v=D0WhSW0phx4).  As  training  and  operations 
increasingly  merge,  due  largely  to  the  increase  in  virtualized  operations,  the  ability  to  design, 
prototype,  test,  iterate,  and  deploy  in  mixed  reality  will  become  increasingly  important.  This 
has  significant  overlap  with  the  needs  and  desires  for  MBSE,  and  as  such,  that  synergy  can 
hopefully  be  amplified  moving  forward. 


4.2.5  Data  Structures  and  Reasoning 

From  the  overall  integration  of  graphical  CONOPS  to  the  other  modeling  and  simulation 
environments,  we  need  more  information  about  the  underlying  data  and  information  that  is 
captured.  Given  that  this  was  the  first  iteration  of  ESP  design,  ICT  went  through  a  series  of  rapid 
prototypes  to  tease  out  both  conceptual  framework  as  well  as  concrete  issues  that  arise  during 
the  design  of  the  ESP  system.  Inevitably  that  types  and  amount  of  data  tracked  is  a  tradeoff 
based  on  inherent  capabilities  of  the  game  engine,  performance,  and  particular  the  ability  to 
track  user  actions. 

In  particular  ICT  was  interested  in  trying  to  understand  user  intent  within  the  game.  ICT  firmly 
believes  that  current  game  metrics  are  inadequate  to  assess  why  a  user  takes  certain  actions 
within  the  game/simulation.  This  data  and  knowledge  is  less  critical  for  entertainment 
experiences,  but  is  critical  when  considering  that  ESP  is  focused  on  innovation  and 
understanding  a  wide  variety  of  issues  around  design  and  implementation.  As  such  ICT  spent 
time  experimenting  with  various  biometric  markers  and  sensors  in  addition  to  more  tradition 
game  metrics.  It  is  clear  that  true  "next  generation"  game  analysis  will  be  some  combination  of 
in-game  data  aggregation,  manipulation  and  visualization,  along  with  a  robust  "user  model" 
where  each  individual  user  is  tracked  for  not  only  a  history  of  their  in-game  actions  but  also 
they  emotional  and  mental  state. 

It  is  important  to  remember  that  these  initial  ESP  prototypes  are  just  that  -  prototypes  to 
explore  the  problem  space  and  understand  how  ESP  can  actually  be  done  effectively.  In  future 
versions  of  this  report  additional  detail  on  the  data  will  be  included. 


4.2.6  Metadata  and  Metamodel  for  Graphical  CONOPS 

We  received  information  about  the  underlying  metamodel  of  the  information  that  can  be 
captured,  regardless  of  the  domain,  and  the  methods  that  would  be  used  to  ensure  that 
information  is  fully  captured.  We  hope  this  information  would  be  mapped  to  the  Information 
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Model  (UCOO)  and  be  provided  as  input  to  UC02.  In  addition,  we  are  interested  in  how  the 
parameters  of  simulation  entities  can  be  used  in  MDAO  (UC03). 

■  ICT  will  try  to  help  Stevens  classify  any  data  and  analysis  captured  from  ESP  so  it  can  be 
sent  to  an  Information  Model  to  inform  decisions.  It  might  be  possible  for  ICT  to  also 
help  inform  how  our  data  and  analysis  could  best  be  visualized  to  inform  other  parties. 

■  USC  began  working  with  Tobii  Eye  tracker  and  found  how  to  capture  eye  tracking  data 
to  coincide  with  gameplay  from  CounterStrike. 

■  USC  began  talking  with  Army  Game  Studio  (AGS)  to  design  both  data  schemas  and  data 
capture/retrieval  from  AGS's  Operation  Overmatch  (ESP)  for  future  analysis. 

■  USC  began  working  with  ARDEC  on  Game  Engine  integration  with  OneSAF. 

This  is  ongoing  research,  but  the  following  provides  a  list  of  some  lessons  learned  and  tradeoffs: 

■  Memory  (and  human  Readability) 

o  We  chose  to  prioritize  minimizing  memory  over  time 

•  We  only  recorded  continuous  parameters  when  the  delta  between  the  last 
recorded  value  and  the  current  value  exceeded  some  threshold  -  fewer  data 
points  if  a  player/entity  was  sitting  still  but  added  complexity  for  parsing/playing 
back  the  data. 

o  We  could  have  made  other  choices,  but  given  that  this  protocol  was  intended  for 
transmission  over  the  network,  we  maintain  that  a  light,  memory-optimized 
protocol  was  the  right  choice 

■  Design  time  vs.  Post-processing 

o  We  chose  to  save  low-level  data  (position,  orientation,  field  of  view)  with  an  eye 
towards  building  higher-level  features  by  post-processing. 

•  For  example  visibility  -  "entity  0x123  has  entity  0x234  in  its  field  of  view  at 
t+11.2  sec"  -  we  could  compute  this  by  playing  back  the  log  data  and  adding 
additional  events  to  the  event  stream 

o  Post-processing  is  still  pending  do  to  other  priorities 

o  We  were  looking  for  information  about  the  level,  i.e.,  "this  area  is  cover,"  "this  area 
is  a  corner"  -  two  potential  ways  to  get  this: 

•  Post-process  how  players  use  the  level 

•  At  design  time,  program-in  literal  "this  area  is  cover"  etc.  data. 

■  Format 

o  ICT  chose  to  use  JSON  [93]  because  it  is  pretty  succinct  in  comparison  to  something 
like  XML  (which  has  verbose  closing  tags,  etc.) 
o  Organizing  events  by  logging  information  about  entities  (ID,  array  of  timestamped 
events,  event  references) 

ICE  is  working  to  define  a  taxonomy  of  the  data  for  a  post  exercise  analysis  using  JSON.  This  will 
be  important  to  characterize  the  information  model  associated  with  graphical  CONOPS. 
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4.3  Unity  Comparison 

While  both  of  the  CONOPS  scenario  that  use  the  Unity  gaming  engineer,  the  concept  described 
in  4.1  is  an  experiment  on  simulating  drones  being  used  for  surveillance  as  well  as  drones  being 
used  as  a  counter-UAS.  This  allows  a  user  to  play  around  with  drone  characteristics  to  see  how 
it  changes  their  efficacy,  even  as  the  drone  movement  is  automated. 

The  ICT  experiments  with  the  ESP  hover  pallet  vision  piece  investigate  the  pallet  as  a  viable 
option  to  transport  material  across  short  distances  while  bypassing  any  local  dangers  and 
obstacles  quickly.  This  only  allow  different  choice  of  route,  however  the  ICT  pallet  movements 
are  controlled  by  the  user,  as  well  as  its  weapon  systems. 

The  drone  automation  (Section  4.1)  allows  for  a  more  controlled  analysis,  and  we  would  further 
like  to  integrate  with  Simulink-based  control  from  UC05.  The  drone  automation  system  does 
allow  the  user  to  experiment  with  different  properties  of  the  drones  by  allowing  the 
adjustment  of  Length,  Width,  Height,  Drag,  Mass,  Propulsion,  and  battery  size.  The  ESP 
Multiplayer  and  Vehicle  Tuning  Demo  also  allowed  the  user  to  change  similar  properties  such  as 
Engine  Torque,  Max  RPM,  Leg  Length,  Leg  Rotation,  Battery  Size.  The  ESP  exposed  common 
vehicle  properties,  but  also  purposefully  exposed  certain  properties  that  highlighted  specialties 
of  each  vehicle  (i.e.  Spider  tank  could  be  armored,  yet  crippled  spatial  awareness),  so  that  we 
could  encourage  more  experimentation  and  emergent  gameplay  that  could  arise. 

These  approach  are  different  and  illustrate  why  the  same  underlying  platform  can  support 
different  types  of  analysis.  The  drone  automation  system  currently  relies  on  small  quick  single 
player  scenes/experiments  where  they  try  to  focus  on  developing  UAS  capabilities  in 
incremental  iterative  steps.  It  was  developed  rather  quickly,  and  has  been  incrementally 
evolved.  In  contrast,  the  ESP  development  process  does  have  incremental  iterative  steps,  but 
allows  for  more  emergent  and  immersive  gameplay  in  our  multiplayer  setup  that  not  only  tests 
capabilities,  but  also  techniques  using  the  equipment.  This  makes  the  ESP  more  approachable 
for  the  end  user,  such  as  the  soldiers. 

The  next  two  approaches  discussed  in  Sections  4.4  and  4.5  are  also  very  different,  yet  very 
complementary  to  the  Unity-based  approaches. 


4.4  Simulation  Technologies  for  Graphical  CONOPS  (Grogan  View) 

Graphical  CONOPs  engages  stakeholders  in  an  interactive,  immersive  environment  to  develop  a 
CONOPS  [47]  [97]  [115].  It  aims  to  improve  communication  between  users  and  developers  by 
providing  a  common  platform  on  which  to  express  issues,  similar  to  the  concept  of  a  single  text 
in  negotiation  [140]. 

Another  element  of  this  research  is  investigating  the  use  of  standard  simulation  technologies 
for  graphical  CONOPS.  Standards  are  crucial  to  enable  interoperability  and  data  exchange 
across  model  boundaries.  The  two  most  common  standards  for  distributed  simulation  are  IEEE 
Std.  1278  Distributed  Interactive  Simulation  (DIS)  [84]  and  IEEE  Std.  1516  High  Level 
Architecture  (HLA)  [85].  DIS  defines  common  data  structures  (protocol  data  units,  PDUs)  which 
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are  exchanged  between  simulation  members  in  real  time.  HLA  defines  a  common  application 
programming  interface  (API)  to  a  runtime  infrastructure  (RTI)  which  manages  data  exchange 
and  time  synchronization  among  simulation  federates.  Other  related  standards  include  IEEE 
Std.  1730  Distributed  Simulation  Engineering  and  Execution  Process  (DSEEP)  [86],  SISO  Std.  001 
Real-time  Platform  Reference  Federation  Object  Model  (RPR  FOM)  [160],  SISO  Std.  007  Military 
Scenario  Definition  Language  (MSDL)  [161],  and  SISO  Std.  011  Coalition  Battle  Management 
Language  (C-BML)  [162]. 

In  contrast  to  other  combat  modeling  activities  and  broader  military  operations  research  (the 
typical  application  of  the  above  standards),  graphical  CONOPS  directly  supports  system  design 
activities  and,  as  such,  does  not  incorporate  as  much  detail.  Instead,  it  seeks  to  identify 
fundamental  characteristics  of  the  target  problem.  The  outcome  of  a  graphical  CONOPS  activity 
produces  a  set  of  scenario  parameters  to  describe  the  environment  in  which  a  system  will  be 
used.  In  addition,  we  investigate  a  potential  interface  between  a  SysML  model  and  an 
integrated  mission  model. 

To  support  this  research  another  capability  has  been  created  and  demonstrated.  This  is  a 
simple  scenario  with  UAV,  Counter-UAV  as  a  two-dimensional  model  of  a  two  UASs,  one 
"friend"  and  the  other  "foe,"  with  emphasis  on  distributed  simulation  using  HLA  to  synchronize 
model  state  across  simulators  using  internal  interface  within  the  mission  model. 


HLAobjectRoot 


HLA  Federation  _ * 

Object  Model  (FOM) 


UAV  (keyboard  controlled) 


Counter-UAV  (keyboard  controlled) 


Figure  15.  Mission  Model  using  High  Level  Architecture  (HLA)  to  Enable  Distributed  Simulation 


4.5  Mission  Modeling  Using  High  Fidelity  Simulation  VT  MAK  (Roger  Blake) 

We  have  also  secured  an  academic  license  for  the  VT  MAK  /  VR-Forces  tool  as  a  high-end 
alternative  to  the  two-dimensional  simulation  discussed  in  Section  4.4.  VR-Forces  is  a  high 
fidelity  simulation  environment  that  implements  Computer  Generated  Forces  (CGF)  and  a 
Simulator  Development  Environment  using  a  HLA  framework.  VR-Forces  contains  a  multitude  of 
federate  models  that  can  be  used  to  create  interactive  simulation  environments  to  analyze 
various  situations  and  behaviors  of  desired  scenarios.  We  have  decided  to  use  VR-Forces  as  a 
tool  in  our  research  in  order  to  show  the  effects  of  our  research  and  implementations.  Since 
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each  VR-Forces  federate  model  can  be  communicated  with  using  a  Lua  [101]  scripting  language, 
we  can  change  model  parameters  flexibly.  The  idea  is  that  as  the  design  tools  change  value,  we 
can  theoretically  enter  the  new  design  parameter  values  into  the  simulation  models  to  observe 
the  new  behaviors  within  the  high  fidelity  simulation  scenario.  This  again  provides  another  way 
to  use  MDAO  to  consider  different  optimization  (see  Section  6). 

We  developed  a  demonstration  for  a  simple  UAV  simulation.  This  is  being  expanded  into  a 
counter  UAV  mission.  The  scenario  that  we  demonstrated  was  one  which  included  a  UAV  that 
was  scanning  various  entities  that  it  encountered  as  shown  below  in 

Figure  16.  As  we  continue  to  build  this  scenario,  we  plan  to  include  counter  measures  to  the 
UAV  like  a  Surface-to-Air  Missile  System  also  shown  below.  As  the  UAV  flies  to,  and  around  its 
targets,  nearby  Surface-to-Air  Missile  Systems  will  fire  on  the  UAV  if  the  UAV  flies  into  their  kill 
zones  as  demonstrated  by  the  green  RADAR  beams  that  illustrate  the  area  of  coverage  in  the 
Surface-to-Air  Missile  System  shown  below  in  Figure  17  and  Figure  18. 


Figure  16.  UAV  Scanning  Targets 


Figure  17.  Surface-to-Air  Missile  System 
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By  furthering  this  research,  we  hope  to  be  able  to  use  a  publish/subscribe  system  that  is 
implemented  in  the  lolF  that  utilizes  tool  proxies  to  aggregate  design  tool  data  which  can  be 
routed  to  the  recipient  design  tool  though  the  implementation  of  an  ontology  layer,  as 
discussed  in  Section  12.8.  By  doing  this,  we  hope  to  be  able  to  facilitate  the  transfer  of  design 
tool  parameter  data  though  this  network  by  using  the  SWT  layer  as  the  control  point  that 
decides  where  design  parameter  data  is  needed.  We  can  then  link  the  design  parameter  data 
into  the  federate  model  in  our  simulation  to  be  able  to  observe  the  new  model  behavior  in  the 
simulation  environment  based  on  the  new  design  tool  parameter  changes. 


Figure  18.  Surface-to-Air  Missile  System  Area  of  Coverage 


5  Mission  and  System  Capability  Analysis  (UC02) 


A  mission  model  is  a  dynamic  simulation  model  which  evaluates  the  application  of  a  system  in 
the  context  of  a  scenario.  It  simulates  the  system  operation  to  integrate  and  compute  key 
performance  metrics  (KPMs)  and  assess  system  value  over  operational  timescales.  A  mission 
model  may  either  be  controlled  manually  or  executed  autonomously  provided  adequate 
behavior  scripting.  The  system  model  evaluates  static  functional  capabilities  for  a  particular 
system  design.  A  system  model  evaluates  and  optimizes  functional  capabilities  for  a  set  of 
objectives  and  constraints. 

This  section  extends  the  research  discussed  in  Section  4  to  investigate  automatic 
transformation  and  exchange  of  data  between  the  mission  model,  graphical  CONOPS,  and 
system  model.  As  reflected  in  Figure  19,  inputs  to  the  mission  model  include  scenario 
parameters  and  system  functional  capabilities.  KPMs  output  by  the  mission  model  can  be  used 
to  revise  and  alter  scenario  definitions  and  system  designs  as  needed. 


Report  No.  SERC-2017-TR-110 


43 


Date:  August  8,  2017 


Contract  No.  HQ0034-13-D-0004 


Figure  19.  Scenario  parameters  and  functional  capabilities  are  inputs  to  a  mission  model  which  computes 

performance  metrics. 

This  project  uses  an  application  use  case  scenario  to  study  the  MCE  approach  described  above. 
This  notional  case  is  purposefully  simplified  to  allow  rapid  modeling  without  proprietary  or 
sensitive  details,  as  discussed  in  Section  4.4.  The  use  case  scenario  considers  the  conflicting 
operations  between  a  UAV  and  a  counter-UAV  system.  Both  platforms  exist  in  space  and  are 
equipped  with  sensors  and  engagement  devices. 


Figure  20.  UAV  and  Counter-UAV  systems  participate  in  the  scenario. 

Initial  work  has  focused  on  development  of  a  simplified  mission  model  for  the  UAV/Counter- 
UAV  scenario  described  above.  The  mission  model  is  a  Java  executable  which  imports  scenario 
and  system  information  from  external  interfaces.  Context  parameters  defining  the  spatial 
region  are  loaded  from  JSON  file.  System  parameters  defining  the  functional  capabilities  (max 
speed,  etc.)  are  also  loaded  from  JSON  file  and  system  behaviors  can  be  expressed  Lua  scripts 
conforming  to  an  internal  API. 


5.1  Mission  Model  Mapping  to  System  Model 

Paul  Grogan  investigated  creating  a  representation  in  SysML  and  mapping  the  parameters  from 
the  simulation  into  SysML.  We  use  the  mission  model  and  can  extract  out  data  about  individual 
system  elements,  as  well  as  environmental  information.  An  example  of  the  structural  aspect  of 
the  model  is  shown  Figure  21.  Notionally,  there  is  a  logical  mapping  from  the  JSON  to  the  SysML 
model  structure  shown  in  Figure  22. 
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Figure  21.  Mission  Model  -  Structure 


Inherited  variables 
and  types 

Component 
variables  and  types 


Block  definition  diagram 
(model  system  attributes) 


Figure  22.  SysML  Model  -  Structure 

Representing  behavioral  information  in  mission  modeling  can  be  done  with  Lua  [101]  scripts  as 
shown  in  Figure  23.  Lua  is  a  lightweight,  embeddable  scripting  language  (e.g.,  in  Java).  It 
supports  procedural  programming,  object-oriented  programming,  functional  programming, 
data-driven  programming,  and  data  description. 
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Lua  script  input  file 
for  automated 
Counter-UAV 


Calculate  relative 
direction  of  target 


Incrementally  point 
towards  target 


[^jf  C:\Users\pgrogan\git\txse\src\test\resources\tracker.lua  -  Notepad++ 

1  =  1®* 

File  Edit  Search  View  Encoding  Language  Settings  Macro  Run  Plugins  Window  ? 

X 

oSHtt  .  oeUMlPC  *|  t  TiasiE)  GBsHBElj  • 

Htracker.lua  □  J 

math .min (anglel ,  angle2) 


□function  getDeltaAngle (anglel ,  angle2) 

local  delta  =  math .max (anglel ,  angle2) 
if  delta  >  math. pi  then 

delta  =  math.pi*2  -  delta 

end 

return  delta 

end 


Eplif  not  (this  :getTarget  ()  =  nil  or  simulator  :getElement  (this :  getTarget  () ) 
local  target  =  simulator : getElement (this : getTarget ( ) ) 
local  thisAngle  *  this : getHeading ( ) : getAngle ( ) 
local  targetAngle  =  math . atan2 ( 

target : getLocation ( ) : getY ( ) -  this : getLocation ( ) : getY ( ) , 
target : getLocation () :getX()  -  this : getLocation () :getX () ) 

local  deltaHeading  =  math . rad (this : getAngularSpeedO *duration/1000) 
local  deltaTarget  =  getDeltaAngle (thisAngle ,  targetAngle) 

if  deltaTarget  <  deltaHeading  then 
this : setNextHeading (targetAngle) 
elseif  getDeltaAngle (thisAngle  +  deltaHeading,  targetAngle) 

<  getDeltaAngle (thisAngle  -  deltaHeading,  targetAngle)  then 
this : setNextHeading (thisAngle  +  deltaHeading) 

else 

this : setNextHeading (thisAngle  -  deltaHeading) 

end 


Lua  source  File  length  :  996  lines :  27 


Ln :  1  Col :  1  Sel:0|0 


Windows  (CR  LF)  UTF-8 


Figure  23.  Mission  Model  of  Behavior 

In  SysML  behaviors  can  be  represented  in  state  machine  (stm)  or  activity  (act)  diagrams  as 
shown  in  Figure  24.  SysML  behaviors  can  also  be  represented  in  sequence  diagrams  (not  shown 
here).  While  these  are  intuitive  abstractions,  the  diagrams  cannot  easily  be  transformed  to 
scripted  code  (e.g.  Lua  script),  because  they  are  usually  more  abstract  to  facilitate 
documentation;  this  could  notionally  double  the  effort  to  implement  and  completely  document 
the  models. 
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Figure  24.  SysML  Models  of  Behavior 

The  following  lists  some  of  the  challenges  with  the  integration  to  SysML: 
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■  Lack  of  "acceptable"  representations  and  transformation  using  SysML;  we  are  planning 
to  investigate  this  more  deeply  in  UC04 

■  Graphical  diagrams  specified  at  multiple  abstractions 

■  Oriented  towards  concrete  design 

■  Likely  to  be  missing  relevant  mission/scenario  parameters 

■  XMI  is  difficult  to  'query'  for  structural  parameters 

■  Low-level  with  extensive  unique  IDs  difficult  to  interpret/parse 

■  Behavioral  diagrams  cannot  easily  be  transformed  to  scripted  code  (e.g.  Lua  script) 

These  finding  are  different  from  those  of  the  Challenge  Area  #1  effort,  but  yet  with  similarities. 
The  overarching  challenge  is  the  difficulty  of  tool-to-tool  integration.  This  is  again  the  reason  for 
our  belief  that  we  will  need  to  use  interoperability  using  the  underlying  information  model 
(Challenge  Area  #3). 


5.2  Using  Semantic  Web  Technology  for  Mission  Modeling  and  Simulation 

In  support  of  UCOO,  this  use  case  is  being  extended  to  research  the  use  of  centralized  shared 
information  using  the  lolF  and  specifically  the  use  of  SWT  by: 

■  Populating  the  system  model  represented  in  the  SWT  using  sensor  data  from  other 
simulations 

■  Query  the  system  model  (i.e.,  SWT,  SPARQL)  to  retrieve  specified  design  attributes  (e.g. 
retrieve  system  attributes  as  inputs  to  the  mission  analysis) 

■  Store  analysis  results  for  later  use  by  other  modules  (e.g.  store  mission  analysis  results 
for  use  in  downstream  decision  support  modules) 

The  current  research  extends  the  2D  modeling  and  simulation  environments  for  distributed 
simulations  to  integrate  through  the  components  of  the  lolF  as  shown  in  Figure  4.  As  discussed 
in  Section  4.1,  we  demonstrated  this  concept  for  our  sponsors  using  the  lolF  SWT.  As  shown  in 
Figure  26,  we  created  a  simplified  version  of  a  use  case  to  demonstrate  data  exchange,  which 
demonstrates  a  subset  of  the  functionality  of  the  lolF: 

■  UAV  model:  output  system  performance  attributes 

■  C-UAV  model:  output  system  performance  attributes 

■  Mission  model:  evaluate  system  performance  in  context  of  simulated  mission 
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Figure  25.  UC01-UC03  Prototype  Application  Case 

We  created  a  simple  ontology,  not  for  the  purpose  of  illustrating  how  to  develop  a  "proper 
ontology,"  but  more  as  the  basis  for  showing  examples  of  using  SWT  for  interoperability  using 
the  lolF.  The  small  ontology  describes  class  of  shared  information  using  OWL,  object  properties, 
and  data  properties,  as  shown  in  Figure  26.  The  model  instances  corresponding  to  the  red  and 
blue  systems  are  produced  in  RDF,  and  then  added  to  a  triple  store.  SPARQL  queries  retrieve 
and  update  values  to  create  a  dynamic  interaction  through  the  Data  Acquisition  and 
Aggregation  layer  (DAA)  in  conjunction  with  the  SWT  as  shown  in  Figure  27. 
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Figure  26.  Simple  Ontology  for  Experiment  of  Simulation  Integration  the  SWT 


Some  examples  of  the  underlying  details  of  the  information  described  in  the  ontology  are 
shown  below  in  the  Terse  RDF  Triple  Language  (Turtle).  The  Subject-Predicate-Object  triples  are 
easier  to  read  in  Turtle  than  the  underlying  XML.  For  example  ":Attribute  is  a  rdf:type  of  the  owl 
Class."  In  general,  most  user  of  this  type  of  underlying  technology  never  see  this  level  of  detail, 
and  we  refer  interested  readers  to  other  sources  [182]. 

: Attribute  rdf : type  owl: Class  ; 

rdf s : subClassOf  [  rdf : type  owl : Restriction  ; 

owl : onProperty  :hasUnits  ; 
owl : someValuesFrom  xsd: string 

]  , 

[  rdf: type  owl : Restriction  ; 
owl : onProperty  :hasValue  ; 
owl : someValuesFrom  xsd: double 

]  • 

:hasUnits  rdf : type  owl : DatatypeProperty  ; 

rdf s : subPropertyOf  owl : topDataProperty  ; 
rdf : type  owl : FunctionalProperty  ; 
rdfs: domain  : Attribute  ; 
rdfs: range  xsd: string  . 

:hasValue  rdf : type  owl : DatatypeProperty  ; 

rdf s : subPropertyOf  owl : topDataProperty  ; 
rdf : type  owl : FunctionalProperty  ; 
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rdfs: domain  : Attribute  ; 
rdfs: range  xsd: double  . 

: UAV  rdf : type  owl: Class  ; 

rdf s : subClassOf  [  rdf : type  owl : Restriction  ; 

owl : onProperty  :hasMaxSpeed  ; 

owl : qualif iedCardinality  " 1 " A Axsd : nonNegativelnteger  ; 
owlionClass  :MaxSpeed 

]  , 

[  rdf : type  owl : Restriction  ; 
owl : onProperty  :hasTurnRate  ; 

owl : qualif iedCardinality  " 1 " AAxsd : nonNegativelnteger  ; 
owlionClass  : TurnRate 

]  • 

:MaxSpeed  rdf : type  owl: Class  ; 

rdf s : subClassOf  : Attribute  . 

: TurnRate  rdf : type  owl: Class  ; 

rdf s : subClassOf  : Attribute  . 

:hasMaxSpeed  rdf : type  owl : Ob j ectProperty  ; 

rdf s : subPropertyOf  : hasLinearSpeed  ; 
rdf : type  owl : FunctionalProperty  ; 
rdfs: range  :MaxSpeed  . 

:hasTurnRate  rdf : type  owl : Ob j ectProperty  ; 

rdf s : subPropertyOf  : hasAngularSpeed  ; 
rdf : type  owl : FunctionalProperty  ; 
rdfs: range  : TurnRate  . 


SUB  blue/a ngularSpeed 

Figure  27.  Multi-fidelity  Mission  Simulation  using  Semantic  Web  Technology  and  Data  Acquisition  and 

Aggregation 

There  is  a  video  of  a  simplified  version  of  this  demonstration  as  shown  in  Figure  28.  The  video 
was  shown  to  our  ARDEC  sponsors.  In  this  simple  demonstration,  Model  A  publishes  data  to  the 
DAAL  using  its  proxy,  which  inserts  the  data  into  the  triple  store  using  a  SPARQL  query  (note:  a 
SPARQL  query  can  read  or  write  to  a  triple  store).  Model  B  subscribes  to  the  "RedAngularData." 
The  DAAL  subscribe  method  performs  a  SPARQL  query  to  retrieve  the  data  and  send  to  Model  B 
proxy. 
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Figure  28.  Video  Demonstrating  Integration  and  Interoperability  Framework 

The  latest  instantiation  of  the  research  involved  five  of  the  researcher  to  execute  a 
demonstration  as  reflected  in  Figure  29.  This  version  of  the  lolF  uses  two  active  models  and 
passes  published  data  through  the  SWT  layer  before  delivering  the  data  to  the  subscribing 
model.  The  published  data  that  is  passed  into  the  SWT  is  extracted  in  different  units  and  by 
different  name.  The  example  demonstrates  the  ability  of  the  lolF  to  convert  both  units  and 
name,  through  the  following  steps: 


■  SysML  Model  used  to  model  Red  Team  linear  speed 

■  DocGen  transforms  SysML  model  data  to  xml  format 

■  Proxy  A  captures  and  transforms  xml  data  to  RDF 

■  Proxy  A  publishes  red  team  linear  speed  (in  m/s)  to  DAA 

■  Linear  speed  variable  name  and  units  will  not  match  what  is  needed  for  Proxy  B 

■  Mission  Model  Proxy  B  subscribes  to  red  team  linear  speed 

■  DAA  handles  publish  and  subscribe  from  proxies 

■  SWT  resolves  the  differences  in  the  variable  naming  of  Red  Team  linear  speed  and  also 
the  units 

■  When  Proxy  A  (DocGen)  publishes  a  new  linear  speed  then  the  DAA  initiates  a  request 
to  the  SWT  to  get  the  needed  information  for  the  subscribers  of  that  data  (Mission 
Model)  and  sends  the  updated  information  to  the  subscriber  (Mission  Model) 

■  DAA  stores  RDF  instance  data 

■  For  the  Demo,  the  team  manually  changed  SysML  model's  linear  speed  and  re-ran 
Mission  Model  simulation  to  demonstrate  automated  propagation  of  data  change 
through  system 
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Figure  29.  Integrating  System  Model  Data  through  SWT  to  2D  Simulation 

6  Multidisciplinary,  Design,  Analysis  and  Optimization  (UC03) 


This  use  case  investigates  the  methods  to  trace  capabilities  to  the  relevant  design  disciplines 
and  perform  cross-domain  analyses  through  Multidisciplinary  Design  Analysis  &  Optimization 
(MDAO)  for  problem  and  design  tradespace  analyses.  In  addition,  to  characterizing  elements  of 
the  framework,  cross-domain  relationships,  but  also  characterize  the  methods  used  to  support 
MDAO  in  a  tool  independent  manner. 

MDAO  is  an  approach  for  calculating  optimal  designs  and  understanding  design  trade-offs  in  an 
environment  that  simultaneously  considers  many  types  of  simulations,  evaluations,  and 
objectives.  For  example,  when  designing  a  vehicle,  there  is  typically  a  trade-off  between 
maximizing  performance  and  maximizing  efficiency,  where  calculating  either  of  these  objectives 
require  multiple  disciplinary  models  (geometry,  weight,  aerodynamics,  propulsion).  MDAO 
prescribes  ways  to  integrate  these  models  and  explore  the  necessary  trade-offs  among  the 
objectives  to  make  a  design  decision.  While  the  theoretical  foundations  of  MDAO  are  well- 
established  by  academics,  a  number  of  barriers  to  practical  implementation  exist.  Chief  among 
these  is  the  lack  of  model  integration,  which  prevents  designers  of  one  subsystem  from  easily 
assessing  how  changing  a  design  variable  affects  the  results  of  other  subsystems'  models  or 
simulations.  The  overarching  objective  of  this  use  case  is  to  understand  these  challenges  and 
develop  recommendations  for  overcoming  them  and  effectively  applying  MDAO  to  add  value  in 
a  large,  distributed,  organization  such  as  ARDEC. 

As  illustrated  by  some  of  the  examples  in  UC01  and  UC02,  we  can  extract  the  key  parameters  in 
these  various  mission  and  system  simulations.  These  parameters  are  fundamental  to  the  MDAO 
workflows.  We  need  to  combine  those  parameters  for  different  elements  of  a  workflow,  but  we 
must  also  characterize  our  key  performance  parameters  (KPP);  for  example,  a  surveillance  UAV 
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range  or  endurance  would  be  KPPs.  These  KPP  are  modeled  as  the  outputs  from  running  the 
MDAO  through  different  optimizations.  The  other  aspect  of  the  method  involves  identifying  the 
constraints  that  must  be  characterized  with  respect  to  KPPs  (i.e.,  outputs)  with  respect  to 
selected  inputs.  As  discussed  in  Section  9,  we  believe  that  the  decision  framework  (see  Figure  1) 
use  case  UC06  provides  a  methodological  approach  to  identify  the  KPPs. 


6.1  MDAO  Objectives 

More  specific  objectives  include: 

■  Assessing  the  impacts  of  individual  design  changes  on  system  capabilities 

■  Supporting  early-phase  (conceptual  design),  system-level  trade-off  analysis  using 
previous  evaluation  results  from  existing  models 

■  Develop  strategies  to  transform  the  contracting  process  so  that  requests  for  proposals 
(RFPs)  can  be  designed  more  flexibly  toward  value-based  (rather  than  target-based) 
design 

In  pursuit  of  these  objectives,  the  research  activities  entail: 

■  Develop  generic  multidisciplinary  models  of  an  UAS,  including  analyses  of  the  geometry, 
structure,  aerodynamics,  propulsion,  and  performance  capabilities,  to  be  used  as  an 
example  case 

■  Explore  using  systems  representations  (e.g.,  SysML,  Domain  Specific  Models)  to  map  all 
inputs  (parameters  and  variables)  and  outputs  (objectives,  constraints,  intermediate 
parameters)  among  the  individual  models 

■  Conduct  trade  studies  on  the  UAS  design  using  established  approaches  and  tools  for 
MDAO,  exploring  different  approaches,  tools,  and  visualization  techniques  to  most 
effectively  display  information  and  uncertainty  for  decision-makers 

■  Explore  ways  that  previous  trade  study  results  on  detail-phase  product  design  can  be 
useful  toward  new  conceptual  design  of  products  with  varying  mission  capability 
requirements 

■  Work  with  ARDEC  project  leads  to  understand  the  barriers  to  implementing  this  type  of 
MDAO,  culturally  and  practically/theoretically 

■  Explore  more  general  ways  to  map  and  coordinate  subject  matter  experts  (SMEs)  and 
data,  models,  and  meta-models  for  improved  (1)  requirements  setting  for  RFP  or 
CONOPS,  and  (2)  value-driven  design 

Interfaces  with  other  sub-tasks  include: 

■  Explore  ways  to  more  seamlessly  associate  parameters  from  mission  and  system 
modeling  and  simulation  for  UC01  and  UC02 

■  Receiving  and  using  model  structures  from  "Use  Model  Based  Engineering",  "Develop 
Information  Model",  and  "Create  System  Models"  portions 

■  Feeding  and  matching  capabilities  and  needs  with  the  "Research  Mission  and  System 
Operational  Capabilities"  and  "Research  Graphical  CONOPS"  portions  of  the  project,  as 
well  as  the  "Research  Decision  Framework"  portion 
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■  Investigate  how  MDAO  outputs  can  be  further  used  to  calibrate  mission  and  system 
modeling  and  simulation 

■  Investigate  if  MDAO  can  be  used  to  formalize  the  Assessment  Flow  Diagram  (AFD)  for 
the  Decision  Framework  (UC06) 

One  of  the  objectives  of  this  project  is  to  leverage  the  most  powerful  tools  that  are  often  used 
by  industry  as  well  as  government  organization.  We  have  secured  academic  licenses  to  Phoenix 
Integration's  ModelCenter  [138].  Further,  while  research  to  date  examines  the  use  of  MDAO  at 
the  systems  level.  We  have  received  additional  academic  licenses  to  ModelCenter  to  investigate 
the  use  of  MDAO  at  the  mission  and  subsystem  levels. 


6.2  MDAO  Methods 

Using  tools  like  ModelCenter,  we  have  investigated,  demonstrated  and  described  methods  for 
apply  such  tools,  and  also  identify  the  relevant  research  questions  in  the  context  of  those 
advanced  tools.  For  example,  the  steps  for  an  MDAO  method  may  be  characterized  as: 

■  Describe  a  workflow  (scenarios)  for  a  KPP  (e.g.,  range,  notionally  similar  to  surveillance 
time) 

■  Determine  relevant  set  of  inputs  and  outputs  (parameters) 

■  Illustrate  how  to  use  a  Design  of  Experiments  (DoE)  and  use  analyses  such  as  sensitivity 
analysis  and  visualizations  to  understand  the  key  parameter  to  use  with  optimizations 

■  Illustrate  Optimization  using  solvers  with  key  parameters  and  define  different  (key 
objective  functions  -  on  outputs)  to  determine  set  of  solutions  (results  often  provided 
as  a  table  of  possible  solutions) 

■  Use  visualizations  to  understand  relationships  of  different  solutions 

A  number  of  methods  can  be  applied  to  formulate  multidisciplinary  optimization  problems, 
develop  useful  surrogate  models,  and  calculate  optimal  and  Pareto-optimal  solutions. 
Optimization  problems  can  be  formulated  with  a  number  of  different  objectives  by  converting 
some  objectives  to  targets  or  constraints,  summing  the  objectives  with  value-based  and  unit- 
consistent  weighting  schemes,  or  multiplying  and  dividing  objectives  by  one  another.  Surrogate 
models  are  often  used  to  quickly  simulate  the  behavior  of  a  more  computationally-intensive 
simulation  model,  and  some  common  methods  include  interpolation,  response  surface  using 
regression  models,  artificial  neural  networks,  kriging,  and  support  vector  machines.  Finally, 
numerical  optimization  can  be  performed  using  a  number  of  different  algorithms  and 
techniques,  including  gradient-based  methods,  pattern  search  methods,  and  population-based 
methods.  For  each  of  these,  different  techniques  have  been  found  to  be  more  suitable  to 
different  applications,  and  part  of  this  research  directive  will  be  to  identify  and  demonstrate  the 
best  tools  for  this  MCE  architecture. 


6.3  Integrations  with  Related  Tasks 

While  the  theoretical  foundations  of  MDAO  are  well-established  by  academics,  a  number  of 
barriers  to  practical  implementation  exist.  Chief  among  these  is  the  lack  of  model  integration, 
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which  prevents  designers  from  easily  assessing  how  changing  one  design  variable  affects  the 
outputs  from  different  models  or  simulations.  Through  this  project,  and  the  creation  of  an  MCE 
architecture  that  follows  a  Single  Source  of  Truth  and  a  consistent  ontology,  we  will  be  able  to 
leverage  MDAO  techniques  in  the  design  decision-making  process.  From  an  academic 
perspective,  the  major  contributions  will  be  to  build  a  roadmap  for  integrating  MDAO  practices 
into  complex  existing  and  new  organizational  structures. 

A  solid  framework  for  MDAO  can  enable  multi-objective  optimization,  showing  product 
developers  how  different  design  objectives  compete  with  one  another.  For  example,  we  know 
that  improving  an  objective  like  "minimize  weight"  typically  requires  a  sacrifice  in  the  objective 
to  "maximize  power."  The  magnitude  of  that  improvement-sacrifice  relationship,  which  often 
involves  different  units  and  requires  human  judgement  to  make  a  mission-appropriate  decision, 
can  be  revealed  by  combining  different  simulation  models,  surrogate  models,  and  optimization 
routines.  As  this  may  involve  balancing  a  large  number  of  objectives,  one  of  the  key  challenges 
is  in  visualization  of  the  results  to  enable  informed  decision-making.  This  fits  into  all  five  tasks  of 
the  project,  as  the  entire  information  architecture  must  be  built  to  support  cross-disciplinary 
analysis,  and  specific  tools  and  techniques  can  be  integrated  and  tested  at  different  stages  of 
the  transformation. 


6.4  MDAO  UAV  Examples  and  Use  Cases 

Demonstration  covering  several  of  the  objectives  have  been  presented  in  several  working 
sessions  as  well  as  several  bi-weekly  status  meetings.  The  demonstrated  workflow  shown  in 
Figure  30  was  developed  using  ModelCenter,  or  in  conjunction  with  SysML  and  the  MBSE 
Analyzer  that  provides  an  integration  from  MagicDraw  SysML  models  to  ModelCenter.  This 
section  provides  a  summary  of  the  evolving  use  of  MDAO  and  different  workflows. 


6.4.1  MDAO  Example  for  Fixed  Wing  UAV 

The  first  demonstration  covered  several  aspects  of  the  objectives  discussed  in  this  section, 
including: 

■  Describe  and  execute  a  workflow  analysis  of  UAS  capabilities  (e.g.,  range,  velocity,  and 
fuel  consumption) 

■  Map  relationships  among  parameters  (inputs/outputs)  in  disciplinary  models 

■  Illustrate  use  of  Design  of  Experiments  (DoE),  sensitivity  analysis,  and  visualizations  to 
understand  capability  relationships/trade-offs 

■  Optimize  using  different  solvers  to  find  sets  of  Pareto-optimal  solutions 

■  Take  advantage  of  previous  model  analyses  for  use  in  early-phase  design  with  new 
mission  capability  requirements 
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Figure  30.  MDAO  Example  Workflow 


As  shown  in  Figure  31,  the  Pareto  frontier  (Pareto  optimal  set)  shows  the  trade-off  between 
range  and  propulsion.  The  blue  points  show  the  Pareto  frontier/non-dominated  solutions.  The 
Pareto  frontier  was  calculated  using  a  bi-objective  optimization  using  NSGA-II  algorithm  to: 

■  Maximize  range 

■  Maximize  propulsion 

■  Given  5  design  variables 
o  Wing  area  (ft2) 

o  Wing  span  (ft) 
o  Altitude  (ft) 
o  Speed  (knots) 
o  Efficiency  factor 

These  results  reflect  on  how  much  range  one  would  have  to  give  up  in  order  to  increase  the 
propulsion  by  some  amount.  Based  on  the  current  set  of  equations  characterized  in  the 
workflow,  the  sensitivity  analysis  shown  in  Figure  32  indicates  that  the  wing  area  is  the  variable 
that  exhibits  the  clearest  trade-off.  The  wing  span  has  the  largest  effect  on  range,  but  does  not 
present  a  trade-off  between  these  objectives. 
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Figure  31.  Pareto  frontier  (Pareto  optimal  set)  Shows  Trade-off  Between  Range  and  Propulsion 
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Figure  32.  Sensitivity  of  Objectives  to  Design  Variables 


6.4.2  Extending  the  MDAO  UAV  Example  1 

Brian  Chell  is  a  new  PhD  student  working  with  Steven  Floffenson.  Brian  has  produced  a  number 
of  updates  to  the  initial  model.  The  efforts  produced  alternative  workflows  that  leverage  other 
types  of  solvers  for  different  aspects  of  the  problem  including  multi-physics  problems.  For 
example,  one  of  the  first  steps  looked  at  bring  SolidWorks  [165]  into  ModelCenter  as  shown  in 
Figure  33.  This  provides  a  way  to  bring  in  detailed  geometries. 
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Figure  33.  MDAO  Workflow  with  SolidWords  Computer  Aided  Design  Model 


There  were  a  few  challenges  with  the  more  complicated  geometries,  as  well  as: 


■  Open-source  geometry  validity  is  questionable 

■  Model  variables 

o  Most  SolidWorks  files  found  so  far  do  not  import  variables  into  ModelCenter 
automatically 

o  We  assume  that  we  can  set  the  variables  within  SolidWorks,  but  this  might  be  more 
difficult  because  manually  setting  values  may  not  align  structures  (e.g.,  wing  connect 
to  fuselage  to  meeting  correct) 

■  More  complex 

o  Computations  solver  (e.g.,  CFD)  take  longer  to  run  on  the  laptops  provide  to 
students 


This  has  led  to  the  following  investigations: 

■  Equation-based  models  derived  from  the  model  shown  in  Section  6.4 
o  Uses  DLR  Unmanned  Combat  Air  Vehicles  (UCAV)  [100]  parameters 
o  Model  is  fully  operational 

o  Based  on  weight  fractions  that  are  more  scalable,  and  easier  to  change  than  DLR 
UCAV  model 

o  Model  starting  with  payload  weight  vs.  range  vs.  endurance  tradeoffs 
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o  Looking  at  the  potential  to  merge  with  future  Computational  Fluid  Dynamic  (CFD) 
results 

■  Simulation-based  models 
o  Difficulties 

•  Still  problems  with  importing  variables  into  ModelCenter 

•  Very  large  number  of  variables  automatically  imported  (12,000+) 

•  Under  construction 

o  OpenVSP  [133]  vs.  Solidworks  (CFD) 

•  OpenVSP  is  a  parametric  aircraft  geometry  tool 

•  OpenVSP  allows  the  user  to  create  a  3D  model  of  an  aircraft  defined  by  common 
engineering  parameters.  This  model  can  be  processed  into  formats  suitable  for 
engineering  analysis. 

•  OpenVSP  commonly  used  with  ModelCenter 

•  SolidWorks  has  stronger  analysis  capabilities 

•  OpenVSP  is  limited  to  a  standardized  shape  library 

•  SolidWorks  Flow  Simulation  can  handle  turbulence 

•  OpenVSP  CFD  is  most  valid  at  nominal  flight  conditions  (e.g.  low  angle  of  attack) 

•  OpenVSP  should  be  sufficient  for  conceptual  design  phase 

OpenVSP  is  being  used  for  CFD.  It  is  easier  to  use  with  limited  library  of  shapes  of  quadcopters 
and  fixed  wing,  and  can  run  'headless'  (i.e.,  without  GUI)  to  make  computations  less  expensive. 
NASA  has  been  using  this  with  ModelCenter.  The  current  status  is: 

■  Integrated  parametric  geometry  and  CFD  into  ModelCenter 

■  Performing  optimization  and  DOE  to  characterize  model 

■  Trying  to  find  lowest-fidelity  mesh  that  produces  accurate  results 

■  Challenges: 

o  Takes  some  time  to  change  between  different  aircraft 
o  Future  NASA  wrapper  will  make  this  much  easier 

o  High-fidelity  CFD  simulations  are  very  slow;  we  know  it  can  run  much  faster,  because 
we  tested  on  Mark's  computer;  we  have  not  tried  it  on  the  server,  because  we  don't 
have  enough  licenses 

Figure  34  show  the  CFD  results  from  the  same  geometry  under  the  same  flight  conditions  with 
different  fidelity  meshes.  The  simulation  on  the  left  has  a  coefficient  of  lift  many  magnitudes 
higher  than  the  one  on  the  right.  The  next  steps  will: 

■  Investigate  mesh  balancing  accurate  results  and  low  computing  cost 

■  Start  integrating  structural  analysis 

o  First  use  built-in  OpenVSP  outputs  for  wings  modeled  as  simple  beams 
o  Investigate  using  Finite  Element  Analysis  (FEA) 

o  While  this  is  using  an  airplane  in  the  example,  the  concept  is  relevant  to  things  that 
ARDEC  designs  that  must  fly  (e.g.,  quadcopters) 
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Figure  34.  CFD  Mesh  Fidelity  Importance 


6.5  SysML  Integration  to  MDAO  through  MBSE  Analyzer 

This  research  investigated  the  use  of  the  Phoenix  Integration  MBSE  Analyzer  that  provides  a 
way  to  integrate  MagicDraw  SysML  models  with  ModelCenter  for  performing  MDAO  analysis. 
John  Dzielski  who  performed  this  research  primarily  works  in  Matlab,  and  he  used  an  example 
that  was  familiar  to  him  related  to  underwater  super  cavitating  modeling.  The  process  covered 
the  following  steps: 

■  Defining  requirements  models  in  SysML 

o  MBSE  Analyzer  works  by  adding  a  profile  that  includes  a  number  of  stereotypes  to 
MagicDraw 

o  Specify  a  constraint  (='s),  upper  and/or  lower  bounds,  and  units 

■  Properties  are  connected  to  requirements  via  the  satisfy  relationship 

■  Information  is  transferred  to  the  ModelCenter  through  MBSE  Analyzer  plugin  as  shown 
in  Figure  35 

o  Requirements  are  shown  in  the  Margin  column  of  the  plug-in. 
o  The  plug-in  indicates  whether  the  requirements  are  satisfied  or  not  by  a  design 

■  MagicDraw  Plug-In  populates  an  analysis  to  create  a  workflow 
o  Components  correspond  to  constraint  blocks 

o  Constraints  blocks  are  models  or  equations  used  in  par  diagrams 

o  Constraint  parameters  correspond  to  component  variables  in  ModelCenter 

■  Parametric  (PAR)  blocks  are  used  to  indicate  to  ModelCenter  how  to  connect 
component  I/O  (values)  to  model  values 

■  All  of  the  other  types  of  analyses  discussed  previous  can  then  be  applied  in  ModelCenter 
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Figure  35.  Example  of  MBSE  Analyzer  MagicDraw  Plugin  to  Integrate  with  ModelCenter 


The  following  reflects  on  some  of  the  initial  findings  in  his  first  exposure  to  MagicDraw,  SysML, 
and  ModelCenter: 

■  Found  it  difficult  to  learn  SysML 

o  SysML  has  a  lot  of  documentation,  but  MagicDraw  can  be  hard  to  learn  (John 
learned  MagicDraw  without  any  formal  training) 

■  ModelCenter  is  a  little  bit  better 

o  Extremely  flexible,  anything  that  can  be  modeled  in  ModelCenter  can  be  used  a 
constraint 

o  Similar  constraints  will  be  found  in  ARDEC  -  in  specific  armament 
o  MBSE  Analyzer  works  by  adding  a  profile  that  includes  a  number  of  stereotypes  to 
MagicDraw 

■  It  is  easier  to  model  in  SysML  and  use  the  MBSE  Analyzer  to  create  the  ModelCenter 
workflows 

■  ModelCenter  doesn't  understand  generalization  relationships  as  represented  in  SysML 


6.6  MDAO  Next  Steps 

There  are  a  few  additional  tasks  planned  for  this  MDAO  use  case: 
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■  Provide  a  demonstration  of  MDAO  applied  to  the  graphical  CONOPS  use  cases  (UC01) 

■  Continue  to  add  on  simulation-based  model 
o  Propulsion 

o  Internal  components 
o  Payload,  engine,  fuel  tanks 
o  Structural  analysis 

■  Continue  to  look  at  the  MDAO  relationships  to  Design  Structure  Matrix  (DSM) 

■  Investigate  the  use  of  MDAO  for  formalizing  the  Assessment  Flow  Diagram  of  the 
Decision  Framework  (UC06)  and  populating  AAMODAT  (UC10) 


6.7  Formalizing  Assessment  Flow  Diagrams  as  MDAO  Workflow 

For  populating  AAMODAT  (UC10),  we  need  to  collect  all  of  the  elements  of  information.  Mary 
Bone  demonstrated  at  the  third  working  session  how  this  is  feasible.  However,  we  need  to 
determine  how/where  to  collect  all  of  the  information  reflected  Figure  36  from  rigorously 
specified  models.  Matt  Cilli,  believes  this  is  possible. 
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Matt  Cilli  has  done  a  refinement  from  his  book  chapter,  and  created  an  Assessment  Flow 
Diagram  (AFD)  as  shown  in  Figure  37.  The  research  investigates  if  we  can  formalize  the  AFD  as 
an  MDAO  workflow,  which  will  have  a  mapping  into  SysML.  John  starts  with  SysML  and  use  the 
MBSE  Analyzer  to  produce  the  MDAO  workflow.  Initial  assessments  reported  at  working  session 
#5  suggest  that  this  is  possible,  and  we  will  next  attempt  to  use  MDK/DocGen  (UC04)  to  extract 
that  information  from  SysML  and  put  it  into  a  repository  to  be  later  loaded  into  AAMODAT. 
Figure  37  provides  a  basic  conceptualization  for  researching  this  concept: 

■  Can  MDAO  represent  Assessment  Flow  Diagram? 

■  Does  AFD  characterize  needed  MDAO  workflows? 


Key  Performance  Function 
(Key  Performance  Parameter  [KPP]) 


MDAO  Workflow  for  KPP 


Figure  37.  Formalizing  the  Assessment  Flow  Diagram 


6.8  Future  Research  for  MDAO 

At  the  request  of  David  Allsop  from  Boeing,  we  also  connected  a  few  people  from  our  NAVAIR 
visits  to  discuss  the  issue  of  deriving  MDAO  parametrics  from  high-fidelity  models,  or  more 
generally  having  some  type  of  bi-directionality  between  parametric  models  and  higher  fidelity 
simulations  (which  can  "break"  the  parametric  chains).  Dr.  Dave  McCormick  who  runs  the 
MDAO  lab  for  Northrop  Grumman  gave  an  informative  presentation  at  the  April  NDIA  Modeling 
and  Simulation  bi-monthly  committee  meeting  on  some  of  challenges,  which  we  believe  are 
relevant  to  future  research,  such  as: 

■  Rapid  re-parameterization  of  completely  new  concepts 

■  Ability  to  incorporate  static  models 

■  Ability  to  bring  in  static  changes  "underneath"  the  parameterization 

■  Ability  to  incrementally  add  to  parameterization 

■  Ability  to  rapidly  alter  the  sizing  logic  behind  models 
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7  System  Models  and  Model  Based  Systems  Engineering  (UC04) 


This  use  case  applies  MBSE  methods  and  tools  to  the  case  study  examples  and  also  looks  at 
how  metamodels  or  metadata  is  represented  in  the  Information  Model  (UCOO)  to  provide 
traceability  through  the  other  forms  of  modeling  for  UC01,  UC02,  UC03,  UC05,  UC06  and  UC10. 
This  use  case  is  developing  different  variants  of  UAS  system  models  at  both  the  system  and 
mission  level.  We  are  also  interested  in  using  MBSE  using  SysML  with  MagicDraw  [126]  to 
investigate  benefits  and  synergies  through  OpenMBEE  [132],  as  discussed  in  Section  12.6.  We 
used  the  Model  Development  Kit  (MDK)/DocGen  to  generate  visualization  of  the  requirements 
from  the  AVCE  iMBE  model  provide  by  the  ARDEC  sponsors.  The  use  of  MagicDraw  also  allows 
for  integration  to  ModelCenter  through  MBSE  Analyzer,  as  a  means  for  modeling  system 
constraints  in  SysML  and  integrating  with  MDAO  as  discussed  in  Section  6.5. 


7.1  OpenMBEE  and  Model  Development  Kit 

We  have  provided  a  number  of  sessions  to  our  sponsor  on  the  OpenMBEE  that  was  developed 
and  now  is  open-source  by  NASA/JPL.  OpenMBEE  has  been  evolving  over  the  years,  and  we  are 
participating  in  the  OpenMBEE  collaboration  group 

(https://groups.google.eom/d/forum/openmbee/),  which  has  about  115  group  members, 
including  industry  participation  from  Boeing,  Lockheed  and  international  organizations.  We 
believe  it  will  be  an  effective  tool  for  our  research,  but  can  also  provide  us  with  insights  that 
might  be  beneficial  to  AVCE  iMBE. 

As  shown  in  Figure  38,  OpenMBEE  has  three  main  components:  MDK  -  with  DocGen,  Model 
Management  System  (MMS),  and  the  View  Editor.  DocGen  works  from  a  View  and  Viewpoint 
hierarchy,  which  is  a  type  of  model  embedded  within  a  system  model.  In  the  absences  of  more 
rigorous  checking  such  as  the  NASA/JPL  ontologies  [90],  or  validation  rules  from  in  MagicDraw, 
the  use  of  the  View  and  Viewpoint  hierarchies  can  be  used  to  enforce  some  methodological 
guidelines.  For  example,  after  generating  a  document  using  DocGen,  blank  sections  reflect 
potential  incompleteness  in  the  model.  While  the  generated  documents  can  provide  a  type  of 
specification,  they  are  often  used  first  as  a  means  of  checking  the  view  of  a  model  and  then 
"pushed"  into  the  MMS  where  they  can  be  viewed  through  the  View  Editor,  which  runs  in  a 
standard  browser.  The  View  Editor  allows: 

■  Access  by  person,  roles,  supporting  review 

■  Can  update  information  that  can  be  pushed  back  into  the  model  through  the  MMS 

NASA/JPL  hoped  that  the  process  of  open  sourcing  OpenMBEE  would  encourage  tool  vendors 
to  add  capability  into  the  commercial  tools,  and  to  some  extent  this  has  occurred.  The  updates 
created  by  NASA/JPL  improve  the  practice  of  modeling.  Details  are  provided  on  Github: 

https://open-mbee.github.io/. 
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Model  Development  Kit/DocGen 
View  and  Viewpoint  Hierarchy 


7.2  Model  Development  Kit  and  DocGen 

Benjamin  Kruse  has  provided  a  number  of  talks  and  demonstration  covering  the  following 
topics: 

■  Concepts  for  DocGen  as  architecturally  represented  in  Figure  39 

■  View  and  Viewpoint  Hierarchy 

■  Workflows 

■  Best  Practices  and  considerations 

■  Model  Findings  and  System  Reasoner  supported  by  MDK 

■  Usage  &  Purpose 

o  Extracting  information  for  various  stakeholders 
o  Demonstrated  example  for  AVCE  iMBE 
o  Demonstrated  example  for  UAV 
o  Demonstrated  example  for  NAVAIR  Surrogate  Pilot 
o  Thirty  Meter  Telescope  models  has  a  number  of  example: 

https://github.com/Open-MBEE/TMT-SysML-Model/tree/master/Presentations 
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Figure  39.  Concepts  for  DocGen 

The  basic  concepts  of  a  View  and  Viewpoint  hierarchy  are  shown  in  Figure  40.  There  is  a  profile 
for  DocGen,  which  includes  «Document»,  «view>,  and  «viewpoint».  A  Document  contains 
one  more  Views.  A  View  exposes  Model  Content,  and  conforms  to  a  Viewpoint.  A  Viewpoint  is  a 
special  type  of  profiled  activity  diagram,  as  shown  in  Figure  41  that  provides  a  modeling 
language  for  extracting  information  from  the  exposed  view.  While  this  capability  was  developed 
to  "generate  documents"  or  visualizations  from  a  model,  we  believe  that  it  can  be  used  for 
other  purposes: 

■  Use  concept  to  extract  parametric  values  for  translating  into  Monterey-Phoenix 
'language'  -  related  to  RT-176 

■  Use  concept  to  extract  workflow  information  to  support  the  Assessment  Flow  Diagram 
as  discussed  in  Section  6.7 
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Domain  Model  2 

Figure  40.  Concepts  of  View  and  Viewpoint  Hierarchy 


Figure  41.  Simple  Viewpoint  Example 

There  are  a  few  considerations  and  best  practices  for  developing  view  and  viewpoint 
hierarchies  for  use  with  DocGen 

■  There  a  number  of  pre-defined  viewpoints,  so  review  those  provided  in  the  profile  to 
understand  what  is  available,  and  to  provide  guidance  in  making  custom  viewpoints 

■  Expose  model  elements  that  align  with  viewpoints  and  vice  versa 
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o  Required  data  must  exist  in  model  (e.g.  traceability  links  between  elements) 
o  Consistent  model  structure  makes  data  accessible  (e.g.  nested  package  structures  or 
existing  diagrams  at  expected  position) 

■  Ordering  of  sections/views 

■  Order  of  sections/views  conforms  to  order  of  a  set  of  part  properties  as  reflected  in 
Figure  42,  which  shows  partial  representation  of  View  and  Viewpoint  hierarchy  for  AVCE 
iMBE  (DocGen  plugin  only  displays  it  through  numbered  naming) 

o  Create  sub-chapters  through  nested  views  to  reduce  change  impact 

■  Data  representation 

o  Produce  SysML  matrixes  only  as  images  or  tables 
o  There  are  issues  to  export  simulation  plot  data 

■  There  is  a  simulation  capability 

o  Expected  use  for  web  editor  (e.g.  to  recalculate  values) 

o  Execution  of  simulation  within  SysML  during  report  generation,  not  working  as 
expected 

■  Viewpoints  can  be  described  with  the  Object  Constraint  Language  (OCL)  (as  opposed  to 
the  activity  diagram  language) 
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package  Doc  Gen  [  |jg]  AVCE  Report  ]j 


Figure  42.  Partial  Representation  of  View  and  Viewpoint  Hierarchy  for  AVCE  iMBE  Model 


7.3  System  Model  in  SysML 

We  are  also  using  MBSE  to  model  our  project,  as  reflected  in  the  initial  use  cases  shown  in 
Figure  3.  We  are  developing  UAV  examples,  both  for  this  project  as  well  as  for  our  NAVAIR 
research.  We  plan  to  leverage  models  between  the  projects,  where  possible.  For  example,  as 
shown  in  Figure  43,  the  system  domain  shows  the  various  elements  associated  with 
surveillance,  which  is  shown  in  a  Block  Definition  Diagram  (BDD).  We  will  elaborate  on  parts  of 
this  domain  that  map  back  to  both  mission  and  system  simulation  in  UC01,  UC02,  and  UC03. 
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Figure  43.  Surveillance  System  Domain  Diagram 

We  also  provided  an  example  of  Activity  diagram  of  Mission  Activity  relating  a  Sensor  Platform 
(UAV)  and  its  interactions  with  Communication  Platform(s)  as  shown  in  Figure  44  [168].  Note 
that  this  concept  is  presented  from  a  logical  perspective  and  shows  both  control  flow  (dash 
lines),  and  data  flow  (solid  lines);  this  activity  diagram  also  shows  swim  lanes  that  illustrate  the 
different  partitioning  of  the  activities.  NOTE:  these  are  all  examples. 
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activity  Mission  Activity!  rj  Mission  Activity  ] 


Figure  44.  Mission-level  Activity  Diagram  with  Swim  Lane  Partitions 


We  can  further  refine  the  model,  and  we  also  have  examples  that  are  based  on  a  product  family 
of  UAV  being  developed  by  our  research  collaborator  Dr.  Russell  Peak  under  RT-170  that 
include: 

■  Rotor  UAV  2.1  portfolio  effectively  completed 

o  Includes  optical  camera  option  to  original  package  delivery  UAV  squadron 
o  Includes  physics  calculations  via  SysML  parametrics  (par) 
o  Includes  behavior  simulation  via  SysML  state  machine  (stm)  /  activity  (act)  / 
parametrics  (par) 

■  Fixed-wing  UAV  0.1  portfolio  initiated 
o  Inspired  by  fixed  wing  surveillance. 

o  Applying  ~same  approach  as  for  rotor  UAV  portfolio 

■  We  could  use  Dr.  Cilli's  UAV  example 

Some  of  work  in  progress  elements  include  the  system  model  for  the  Fixed-Wing  Refueling 
UAV.  These  are  shown  below  in  a  SysML  BDD,  which  shows  some  of  the  subsystems  of  the  UAV 
that  include:  propulsion,  fuel,  and  refueling  subsystems. 
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Figure  45.  Fixed-Wing  Refueling  UAV  Extension  to  UAV  Portfolio 

There  are  elaboration  on  some  parameters  of  the  fuel  system  as  shown  in  Figure  46  to  do  some 
analysis  on  the  First-Order  Physics  using  SysML  Parametrics.  A  parametrics  diagram  provides  a 
way  to  describe  constraints  between  parameters.  Add-on  analysis  tools  can  then  be  used  to 
verify  that  the  constraints  are  satisfiable  (i.e.,  not  contradictory).  This  model  is  developed  in 
MagicDraw  [126],  and  uses  some  automation  provided  by  a  MagicDraw  plugin  called  the 
Cameo  Simulation  Toolkit  for  requirement  verification  as  shown  in  Figure  47.  For  example,  the 
result  of  pass/fail  on  a  constraint  can  be  traced  directly  back  to  specific  requirement  object  in 
the  model. 
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Figure  46.  Parametric  Diagram  of  Fuel  System 
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Figure  47.  Cameo  Simulation  Toolkit  Verifies  Constraints  Representing  Numeric  Requirements 

We  will  elaborate  on  these  model  to  map  to  UC01,  UC02,  UC03,  and  also  are  investigating  the 
integration  of  other  modeling  capabilities  such  as  Mathworks  [104]  Simulink  and  Matlab  for 
UC05. 

8  Counter  UAS  in  the  Context  of  Model  Based  Engineering  (UC05) 


This  use  case  develops  both  the  Model-Based  Engineering  (MBE)  methods,  the  counter 
Unmanned  Air  Systems  (UAS)  scenarios  and  evolving  approaches  to  Automated  Concurrent 
Engineering,  specifically  related  to  MBE  and  manufacturability.  This  use  case  may  be  split  in  the 
future.  In  the  context  of  working  with  the  physical  representation  of  various  elements  that  are 
characterize  abstractly  in  the  system  model,  in  the  mechanical  and  electrical  space,  we  are 
infested  in  how  MBE  can  improve  the  physical  reliability  through  manufacturing.  Therefore, 
Kishore  Pochiraju  has  discussed: 

■  Representation  Methods,  Model  Frameworks  and  Verification  Tools  for  Cyber  Physical 
Design,  which  are  discussed  more  in  Sections  8.2  and  8.3 

■  Automated  Concurrent  Design  as  discussed  in  Section  8.4 


8.1  Model-Based  Engineering 

We  distinguish  MBE  from  Model-Based  System  Engineering  (MBSE).  Typically,  MBE  involves 
modeling  and  simulation  capabilities  related  to  specific  disciplines,  electrical,  mechanical, 
software,  and  the  potential  use  of  domain-specific  modeling  tools.  Most  importantly,  we  are 
interested  in  how  these  modeling  tools  for  a  specific,  some  of  which  have  analysis  and 
simulation  capabilities,  can  be  integrated  with  mission  and  system-level  modeling  and 
simulation  (e.g.,  UC01  and  UC02),  MBSE  (UC04),  and  MDAO  (UC03).  These  various  type  of 
modeling  capabilities  are  fundamentally  important  for  a  new  class  of  systems  that  are  generally 
referred  to  as  Cyber  Physical  System  (CPS). 
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8.2  MBE  and  Cyber  Physical  Systems  (CPS) 

The  phrase  "cyber-physical  systems",  coined  by  Helen  Gills  [72]  defines  "physical,  biological, 
and  engineered  systems  whose  operations  are  integrated,  monitored,  and/or  controlled  by  a 
computational  core.  Components  are  networked  at  every  scale.  Computing  is  embedded  into 
every  physical  component,  possibly  even  into  materials.  The  computational  core  is  an 
embedded  system,  usually  demands  real-time  response,  and  is  most  often  distributed."  Based 
on  a  2015  National  Academy  of  Science  preliminary  report  [119]  "Cyber-physical  systems  (CPS) 
are  increasingly  relied  on  to  provide  functionality  and  value  to  products,  systems,  and 
infrastructure  in  sectors  including  transportation  (aviation,  automotive,  rail,  and  marine),  health 
care,  manufacturing,  and  electrical  power  generation  and  distribution.  CPS  are  smart, 
networked  systems  with  embedded  sensors,  computer  processors,  and  actuators  that  sense 
and  interact  with  the  physical  world  (including  people),  support  real-time,  guaranteed 
performance  and  are  often  found  in  critical  applications.  Clearly,  the  types  of  UAS  that  are  of 
interest  to  ARDEC  are  CPS. 

Kishore  Pochiraju  presented  his  research  entitled:  Representation  Methods ,  Model  Frameworks 
and  Verification  Tools  for  CPS  Design  in  a  bi-weekly  session.  Some  of  the  challenges  discussed 
involve  uncertain  computation  and  network  delays/latencies  that  can  disrupt  control 
performance  and  plant  stability.  Such  control  performance  is  critical  to  maintain  system 
compositionality  across  these  vary  disciplines  of  a  CPS.  The  integration  of  MBE  tools  with  MBSE 
tools  is  of  particular  interest. 

Another  important  aspect  is  CPS  applications  involve  components  that  interact  through  a 
complex  physical  environment.  Reliability,  security,  trustworthiness  poses  particular  challenges 
in  this  context.  These  CPS  need  to  be  highly  dependable,  reconfigurable,  and  in  many 
applications,  certifiable.  Trustworthiness  must  also  extend  to  the  system  level. 

Andrew  Dawson  joined  the  team  in  January  2017,  and  is  working  with  Kishore  on  UAS 
capabilities  using  Simulink  to  look  at  the  integration  with  SysML  using  MagicDraw  and  the 
integration  with  MDAO.  We  plan  to  integrate  the  concepts  discussed  by  Kishore  with  this  effort. 


8.3  Counter  UAS 

We  have  included  the  counter  UAS  use  case  in  this  section,  because  Kishore  has  other  related 
research  in  his  area  of  expertise.  To  summarize  the  key  objective  for  this  counter  UAS  problem: 

■  Given  a  counter  UAS  system  that  identifies  and  restricts  the  flight  of  a  UAV  in  a  specified 
space 

o  Represent  the  system  using  a  compositional  framework  and  appropriate  models, 
much  of  which  has  been  summarized  in  Section  8.2 
o  Validate 

•  For  example,  analyze  the  abstraction  for  a  requirements  satisfaction 
o  Predict 

•  Performance  degradation  due  to  timings  and  time-delays  in  implementation 
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•  Analyze  the  composability  of  the  components  and  dependence  on  the  system 
performance 

•  Analyze  the  compositionality  of  the  entire  system 
■  Basic  premise: 

o  Watches  the  space  for  a  UAS  (Sensor  Component) 
o  Locates  the  UAS  dynamically  (Localization  component) 
o  Defeats  the  UAS  (Act  component) 

To  put  the  magnitude  of  this  challenge  is  perspective,  Business  Insider  magazine  [37]  reports 
that  the  global  aerial  drone  market  will  reach  nearly  $13  Billion  USD  within  the  next  10  years. 
Nearly  75  percent  of  this  market  is  projected  to  be  defense-related.  The  commercial 
marketspace  is  also  expected  to  develop  into  a  $3  Billion  market  in  the  very  near  future.  UAS 
will  surely  be  ubiquitous  enough  to  be  perceived  as  annoyances  or  worse,  threats,  in  many 
spaces  [54].  The  counter  UAS  technologies  address  the  need  for  defense  against  unwanted  UAS 
in  military  and  public  spaces  and  for  the  enforcement  of  various  regulations  against  drone 
flight. 

The  current  counter  unmanned  air  systems,  depending  upon  the  context,  integrate  various 
"Watch,  Match  and  Catch"  methodologies  [58].  These  systems  include  area  surveillance  to 
detect  the  presence  and  location  of  a  signal  (watch  phase),  match  (associate)  to  a  UAV 
signature,  and  deploy  a  catch  or  defeat  technique  such  as  jamming.  The  watch  phase  can  be 
based  on  passive  detection  of  electromagnetic,  thermal  (IR),  acoustic  signatures  or  active  use  of 
RADAR  [114],  LIDAR,  acoustic  beamforming  [145],  and  optical  tracking  methods.  Match  phase 
entails  the  use  of  library  of  signatures,  machine  learning  methods,  and  physics-based  models  to 
identify  the  presence  of  a  UAV  in  the  surveilled  space  and  also  detect  its  type.  Catch  or  defeat 
[15]  requires  jamming  control  signals  or  physically  affecting  the  flight  of  the  UAV  with  nets, 
projectiles  or  other  UAS.  Use  of  a  particular  method  for  the  catch  phase  may  be  rendered 
infeasible  due  to  safety  requirements  and  the  risk  for  collateral  damage. 

Modeling  is  central  to  all  three  phases.  Watch  phase  technologies  employ  modeling  not  only  for 
enhancing  the  signal  to  noise  ratios,  extracting  localization  information  and  constructing  3D 
representations  of  the  tracked  target,  but  also  for  numerous  other  purposes.  Matching  is 
typically  conducted  based  on  pattern  identification  models  with  the  support  of  datasets.  Catch 
methods  employ  modeling  for  interception  path  planning  and  directing  transmission  antennae 
for  directional  and  selective  propagation  of  jamming  signals. 

Due  to  the  real-time  nature  of  all  the  three  problems,  most  accurate  models  that  have  the 
necessary  computational  efficiency  are  generally  preferred.  Accuracy  versus  time  for  the 
computation  of  a  model-based  solution  is  the  general  trade-off  while  deciding  on  the  best 
algorithm  to  implement  in  each  phase.  The  three  phases  are  typically  distributed  among 
heterogeneous  sub-systems  with  some  pipelining  of  the  tasks.  However,  the  total  time  for 
response  (detect-to-defeat)  will  be  the  sum  of  watch,  match  and  catch  phase  times. 

The  objective  of  this  sub-task  is  to  investigate  the  role  of  modeling  in  both  in  terms  of  the 
effectiveness  (i.e.  accurately  watching,  matching  and  catching)  and  the  performance  (i.e.  total 
response  time  and  availability  times). 
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Three  activities  are  proposed  for  this  use  case: 

1.  Analysis  of  models  used  in  counter-UAS  methodologies:  This  activity  entails  analyzing 
open-military  grade  or  a  commercial  counter  UAS  system  and  mapping  role  and 
performance  of  the  models  in  the  system.  The  expected  deliverable  is  a  broad  state-of- 
the-art  and  gaps  analysis  report. 

2.  Assessment  of  watch-match  phase  models:  The  sub-task  team  will  consider  LIDAR  and 
acoustic-based  detection  technologies  and  the  effectiveness  and  performance  of  the 
models  in  the  watch  and  match  phases  of  the  counter  UAS  problem. 

3.  Limited  field  experimentation:  This  sub-task  team  will  collaborate  in  the  design  and 
conduct  of  preliminary  experimentation  of  an  idealized  counter  UAS  system.  The 
objectives  for  the  experiments  will  be: 

o  Measure  the  performance  of  selected  signal  detection  and  UAV  localization  models 
used  in  watch/match  phases. 

o  Investigate  models  that  enable  selective  defeat  (jamming)  of  one  UAV  flying  in 
homogeneous  or  heterogeneous  swarms. 

With  the  intent  of  isolating  and  measuring  the  role  and  impact  of  the  models,  the 
experimentation  will  be  planned  in  uncluttered  physical  spaces  and  with  known  dynamical 
behaviors  of  the  UAVs. 


8.4  Automated  Concurrent  Design 

Kishore  provide  two  talks  extending  the  first  talk  on  CPS  to  reflect  back  on  how  the  formalism 
and  semantically  rich  information  can  contribute  to  automated  concurrent  design.  The  two 
talks  included: 

■  Knowledge-Based  Product  Design  and  Manufacturing  in  the  context  of  Automated 
Concurrent  Engineering  System  (ACES)  Technologies  to  provide  significant  reduction  in 
product  development  time  and  cost  while  optimizing  the  design  and  its  manufacturing 

o  This  was  prior  research,  but  there  is  a  type  of  metaphor,  where  this  work  in  the 
more  mechanical  space  represented  design  knowledge  to  ensure  manufacturability; 
we  are  attempting  to  do  somethings  similar  in  the  system,  system  of  system,  and 
mission  space 

■  Design  Automation  also  related  to  Automated  Concurrent  Engineering  Approaches 

o  This  particular  research  extended  the  prior  work  by  investigating  the  feasibility  of 
formalizing  the  design  process  to  "provide  a  robot  with  a  set  of  'specification'  to 
provide  a  design  automatically" 

o  This  specifically  formalizes  a  system  as  a  network  of  dependencies  from  requirement 
to  design  controls 

o  Provided  early  approach  to  MDAO  for  tradespace  exploration 
o  Networks  of  formalized  design  information  allow  design  automation  to  proceed 
through  a  search  process  that  can  now  be  enhanced  by  Machine  Learning  and  Deep 
Learning  techniques  and  algorithm 
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8.5  Architecture  and  Prototyping  of  System  Simulation  with  Semantic  Data  Exchange 

The  concept  of  a  network  of  design  dependencies  can  be  characterize  in  SWT.  The  RDF  which 
represent  specific  model  instances,  and  are  aligned  with  an  ontology,  is  a  graph  (network  of 
dependences).  The  concept  is  to  create  "gate  keeper"  tools  that  create/manage  semantics  and 
provide  semantic  data  services  to  simulation  tools.  These  gate  keeper  tools  (two  of  them  to  be 
prototyped)  differentiate  data  store/retrieve  tasks  into  concurrent  add/edit/modify  operations 
on  knowledge  and  data  stores.  The  operations  are  divided  into  knowledge-dependent  (may 
require  negotiations  with  Human/AI  experts),  policy  dependent  (require  reasoners  -  from 
heuristics,  policy  statements),  and  simply  tedious  tasks  (i.e.,  automated  out  -  e.g.  use  of  a 
dictionary/thesaurus  to  check  typos).  The  tools  then  create  appropriate  workflows.  We  also  use 
the  concept  of  "regularized  operations"  meaning  all  operations  on  knowledge/data  stores 
complete  if  the  integrities  of  the  stores  are  maintained. 

Kishore  is  aligning  some  of  his  research  for  semantic  data  exchange  with  our  lolF,  with  the 
objectives  to: 

■  Create  a  "simulation-as-a-Service"  framework  with  multi-physics,  concurrent  and 
concurrent  execution  of  simulations  during  system  architecture  and  design  process. 

■  "On  demand"  and  "As  Appropriate"  trade  simulations  during  various  phases  of  large 
complex  systems  design/integration 

■  Enable  service-discovery,  data-curation  and  tool  interoperability 

■  Generalized  abstraction  for  spatial,  temporal  and  stochastic  fields  with  mapped 
semantics 

■  Framework  requirements: 

o  Generalized  abstraction  for  embedding  simulation  tools 
o  Simulation  concurrency  and  pipelining 
o  Datajnteroperability 
o  Model  abstractions  enabling  substitution 
o  Indexed  simulation  inputs,  outputs,  storage 
o  Abstraction  to  capture  model  use  in  design 
o  Dynamic  data  flow  tracking 

o  Data  model  capable  of  large  (2GB)  data  segments,  access  control,  storage  and 
transport. 

o  Agnostic  to  OS  and  computational  hardware 
o  Open  Application  Programming  Interface  (API) 
o  Support  for  real-time  systems  -  Real-Time  Controller  API 


8.6  MBE  Analysis  For  UAS  Energy  Analysis 

In  order  to  accurately  evaluate  system  performance  as  well  as  design  choice  consequences,  two 
areas  of  battery  system  modeling  have  been  explored  by  Andrew  Dawson  in  support  of  added 
realistic  performance  in  the  quadcopter  UAS  elements  in  the  graphical  CONOPS  (UC01).  The 
two  analyses  include: 
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8.6.1  Mass  to  Energy  Capacity: 

Based  on  the  general  architecture  of  common  battery  systems,  system  mass  was  anticipated  to 
vary  linearly  with  energy  capacity.  Battery  parameters  were  compiled  for  the  catalog  of  systems 
available  from  Gensace  and  Tattu,  which  are  widely  utilized  in  small-scale  UAVs.  For  these 
batteries,  the  expected  relationship  was  confirmed  and  can  be  expressed  as  follows: 

mass  [g]  =  5.472  (capacity  [Ah]  *  voltage  [V])  +  61.87 

The  linear  regression  performed  had  an  R-squared  value  of  0.991.  This  relationship  allows 
battery  mass  to  be  easily  incorporated  into  a  variety  of  performance  and  flight  models. 


8.6.2  Voltage  Variability  During  Discharge: 

This  analysis  examined  the  relationship  between  discharge  levels  and  maximum  available 
power.  Common  battery  systems  specify  a  C-value,  which  is  the  maximum  current  that  the 
system  can  safely  produce.  It  is  typically  expressed  as  the  ratio  of  maximum  current  to  the 
current  produced  when  discharging  over  one  hour  (the  Ah  rating).  Therefore,  a  lOAh  battery 
with  a  C-value  of  10  could  produce  a  maximum  of  100A. 

During  discharge,  battery  systems  experience  reducing  voltage  as  charge  level  decreases. 
Therefore,  for  a  specified  C-value,  the  maximum  power  that  the  battery  can  produce  will 
decrease  along  the  discharge  cycle.  This  is  critical  for  UAV  performance,  because  certain  flight 
or  performance  characteristics  may  degrade  over  the  mission  cycle. 

An  empirical  relationship  between  normalized  discharge  level  (%  of  capacity)  and  voltage  level 
(%  of  rated)  was  determined  based  on  typical  discharge  curve  literature.  This  is  only  intended  to 
demonstrate  the  relationship  and  is  not  fully  representative  of  all  battery  systems.  Two  variants 
were  considered: 

1.  Increasing  C-values  impact  the  amount  of  voltage  sag  during  discharge 

2.  Increasing  C-values  impact  both  the  voltage  sag  and  overall  discharge  capacity 

The  equations  developed  to  describe  these  relationships  can  be  utilized  in  performance  models 
to  determine  the  maximum  available  power  at  any  point  in  the  discharge  cycle. 

9  Decision  Framework  (UC06) 


ARDEC  uses  the  Integrated  Systems  Engineering  Decision  Management  (ISEDM)  Process  to 
improve  defense  acquisition  decision-making.  The  ISEDM  process  addresses  the  pressing  issues 
targeted  by  the  Department  of  Defense's  Efficiency  and  Better  Buying  Power  Initiative  and  the 
7-January-2015  DoDI  5000.02.  A  central  issue  confronted  by  both  the  initiative  and  the 
instruction  was  that  systems  engineering  trade-offs  made  between  capability  requirements  and 
lifecycle  costs  early  in  the  acquisition  process  were  rarely  conducted  and  consequently  realistic 
program  baselines  were  not  established  such  that  associated  lifecycle  costs  of  a  contemplated 
system  are  affordable  within  future  budgets.  Through  the  use  the  ISEDM  Process  and  the  family 
of  synthesized  data  visualization  techniques,  systems  engineers  are  able  to  assess  a  large 
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number  of  system  alternatives  across  a  robust  set  of  competing  objectives  in  the  presence  of 
uncertainty  and  quickly  recognize  important  trends  across  cost,  schedule,  and  performance 
dimensions.  While  the  ISEDM  process  has  been  applied  with  success  to  a  number  of  defense 
research  and  development  projects,  there  are  several  opportunities  for  enhancement  and 
extension. 

There  are  several  objectives  within  this  use  case.  We  want  to  explore  potential  enhancements 
and  extensions  to  the  ISEDM  process  and  the  related  decision  support  tool,  AAMODAT.  This  use 
case  has  been  associated  with  a  new  challenge  area  #5  that  has  expanded  the  objective  to 
include  considering  how  to  integrate  cross  domain  models  with  decision  support  models  while 
executing  ISEDM.  In  addition,  as  shown  in  Figure  1,  the  plan  is  to  extract  information  across  the 
various  domain  models  in  the  underlying  information  model  leveraging  SWT  as  described  in 
UCOO.  Specifically,  we  are  looking  to  demonstrate  the  ability  to  a  create  domain  ontology  that 
align  with  AAMODAT  views  (i.e.,  the  underlying  metamodel  for  AAMODAT).  This  concept  is 
notionally  represented  in  Figure  1.  We  believe  this  capability  to  be  applicable  to  ARDEC,  but 
generally  applicable  to  acquisition  organizations  such  as  NAVAIR. 


9.1  Decision  Framework  Objectives 

More  specific  objectives  include: 

■  Generate  a  library  of  fundamental  objectives  hierarchies:  A  fundamental  objectives 
hierarchy  (and  its  associated  measures)  describes  the  criteria  by  which  the  goodness  of 
each  alternative  is  assessed.  Studies  show  that  the  formulation  of  an  objectives 
hierarchy  is  a  difficult  task  and  is  often  done  incorrectly  -  significantly  impacting 
decision  quality  in  a  negative  way.  The  purpose  of  this  sub-objective  is  to  generate  a 
library  of  thoughtfully  prepared  and  well  vetted  objectives  hierarchies  for  a  set  of 
common  weapon  system  types  such  that  a  systems  engineer  can  use  a  hierarchy  from 
the  library  as  a  starting  point  that  can  be  easily  tailored  for  the  particular  decision  at 
hand. 

■  Develop  a  Decision  Risk  Tracker:  Cilli  [41]  identified  40  potential  pitfalls  associated  with 
systems  engineering  trade-off  analyses  and  through  the  use  of  practitioner  surveys 
measured  the  perceived  likelihood  of  encountering  each  pitfall  and  the  consequence  to 
decision  quality  given  a  particular  pitfall  was  indeed  encountered.  The  purpose  of  this 
sub-task  is  to  develop  a  methodology  to  instantaneously  assess  the  overall  risk  of  a 
systems  engineering  trade-off  analysis  project  and  to  update  the  risk  assessment  as 
known  pitfalls  are  avoided  through  the  use  of  best-practices  through  the  execution  of 
the  study. 

■  Incorporate  a  Decision  Adviser  Feature  into  AAMODAT:  Create  a  context  sensitive  pop¬ 
up  decision  advisor  to  alert  AAMODAT  users  of  best  practices  associated  with  the 
current  process  step. 

■  Add  context  sensitive  best  practices  pop-up  wizard  to  AAMODAT  (avoid  common 
pitfalls) 

■  Create  Objectives  Hierarchy  Library  within  AAMODAT 

■  Enable  Assessment  Flow  Diagram  (AFD)  Auto-generation  in  AAMODAT 


Report  No.  $ERC-2017-TR-n0 


79 


Date:  August  8,  2017 


Contract  No.  HQ0034-13-D-0004 


■  Improve  GUI  for  AAMODAT 

■  Integrate  Data  Visualization  COTS  capabilities  (i.e.  Tableau)  with  AAMODAT 

■  Integrate  Value  Scheme  Elicitation  Tools  with  AAMODAT 

■  Develop  improved  automated  Swing  Weight  Matrix  Generator 

■  Integrate  Conjoint  Analysis  tool  into  AAMODAT 

■  Integrate  DOE  capability  in  AAMODAT  to  generate  run  matrix  for  agent  based  models. 

■  Enable  automated  Design  Structure  Matrix  (DSM)  generation  within  AAMODAT  and  link 
to  IRL  portion  of  schedule  estimator  module 

■  Use  unclassified,  public  releasable,  but  plausible  and  data  rich  problem  (sUAV  case  study 
developed  under  ERS  effort)  to  demonstrate  ISEDM  best  practices  with  new  upgrades 
listed  above. 

■  Solve  same  problem  but  purposely  trip  on  identified  pitfall  to  illustrate  why  ISEDM 
process  that  avoids  pitfall  is  superior. 


9.1.1  Decision  Framework  Methods 

Research  methods  to  achieve  stated  objectives  will  focus  on  the  use  of  new  product 
development  case  studies  approved  for  public  release,  which  is  represented  in  a  book  chapter 
created  by  Matt  Cilli  [42]. 

In  response  to  a  request  from  the  Engineered  Resilient  Systems  (ERS)  program,  ARDEC  is 
generating  a  hypothetical  yet  plausible  case  study  that  can  be  used  to  stimulate  and  focus 
academic  discussion  regarding  systems  engineering  tradeoff  analyses  in  the  context  of  new 
product  development  efforts.  The  case  study  will  possess  elements  of  story  such  as  setting, 
characters,  plot,  conflict,  and  point  of  view  (Omniscient  Limited),  and  theme.  It  will  also 
provide  detailed  narrative  incorporating  many  viewpoints;  involve  ambiguity,  uncertainty,  and 
un-structured  presentation  of  initial  information;  give  rich  description  of  potentially  useful  data 
at  multiple  levels  of  fidelity;  allow  for  multiple  outcomes;  and  be  publically  releasable. 


9.1.2  Integrations  with  Related  Tasks 

The  white  text  within  the  outer  green  ring  of  Figure  48  identifies  systems  engineering  processes 
and  methods  while  the  ten  blue  arrows  represent  the  ten  steps  of  the  analytical  decision 
process.  Interaction  between  the  systems  engineering  processes  and  the  decision  process  are 
represented  by  the  small,  dotted  green  or  blue  arrows. 
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Figure  48.  ISEDM  Process  Summary 


9.2  Using  Semantic  Web  Technologies  to  Formalize  Decision  Framework  for  AAMODAT 

Fundamentally,  if  were  able  to  formalize  the  concept  discussed  in  the  context  of  challenge  area 
#5  by  using  underlying  information  model  UCOO  to  populate  AAMODAT,  we  would  need  for 
formalize  all  of  the  information  and  trace  linkages  through  the  Information  model  back  to  all 
other  related  perspectives  on  the  system  (UC01,  UC02,  UC03,  UC04,  UC05).  These  elements 
would  include: 

■  Objective  hierarchies 

■  Value  functions 

■  Assessment  Flow  Diagrams  (AFDs)  trace  the  relationships  between  physical  means, 
intermediate  measures,  and  fundamental  objectives,  as  discussed  in  Section  6.7 

■  Uncertainties 

Figure  49  shows  the  use  case  refinement  that  has  been  discussed  by  the  team.  Mary  Bone  has 
provided  an  extended  session  at  the  third  working  session  on  a  concept  to  show  how  SWT 
could  support  this  effort  to  populate  AAMODAT. 
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"Wiley:  Trade-off  Analytics:  Creating  and  Exploring  the  System  Tradespace-  Gregory  S.  Parnell."  [Online].  Available: 
http://www.wiley.com/WileyCDA/WileyTitle/productCd-111923753X.html.  [Accessed:  27-Mar-2017]. 


Figure  49.  Decision  Framework  Use  Case  Refinement 
There  have  been  other  developments  toward  that: 

■  Robin  Dillon-Merrill  is  working  on  templates  for  different  type  of  objective  hierarchies 
(e.g.,  portfolio,  product) 

■  Matt  Cilli  has  an  update  to  the  UAV  case  study  [42] 

■  Mary  Bone  walked  through  the  use  case  using  example  to  show  how  to  use  SWT  (UC10) 
to  produce  score  sheet  and  consequence  score  card  for  objective:  reach  areas  of 
interest  quickly 

o  For  demo  purposes,  Mary  used  SWT  to  get  example  data  from  DBpedia  (which  is  a 
crowd-source  effort  to  extract  structured  information  from  Wikipedia  and  make  this 
information  available  on  the  Web) 

o  Created  a  simple  Aircraft  Ontology  &  Properties  for  demo  to  show  semantically  rich 
data  extracted  from  DBpedia  using  SWT  tools  (Protege,  OWL  Viz,  RDF) 
o  More  details  in  UC10 


As  discussed  in  Section  6.7,  we  also  noted  that  the  AFD  is  probably  the  single  view  that  best 
describes  how  the  specific  design  choices  are  made  across  the  product  structure,  and  are 
transformed  into  consequences  across  the  fundamental  objectives  through  an  array  of 
interrelated  models.  Because  of  the  similarity  in  the  AFD  to  MDAO  workflows.  We  are 
researching  ways  to  model  the  AFD  as  an  MDAO  workflow,  because  those  workflows  would 
most  likely  be  related  to  Key  Performance  Parameter  (KPP).  We  had  noted  in  the  past  that  the 
Decision  Framework  would  potentially  support  a  method  for  deciding  on  the  KPPs.  The  AFD 
might  prioritize  the  needed  workflows  to  defined  using  MDAO  (e.g.,  ModelCenter). 
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9.3  Integrated  Models  of  the  Assessment  Flow  Diagram 


See  Section  6.7  which  discusses  the  concept  for  using  MDAO  workflows  to  represent  the  AFD  in 
a  formal  way  that  could  be  automatically  extracted  from  models  to  populate  SWT  for 
automating  the  population  of  AAMODAT.  An  AFD  can  be  used  by  the  lead  Systems  Engineer  to 
organize,  manage,  and  track  assessment  activities  especially  when  used  in  conjunction  with  the 
consequence  scorecard. 
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Figure  50.  Assessment  Flow  Diagram 

10  MCE  Impacts  on  Verification  and  Validation  (UC07) 


There  was  no  explicit  task  to  support  Verification  and  Validation  (V&V),  however  MCE  can 
inherently  produce  information  in  a  more  formal  way  that  can  enable  early  and  continuous 
V&V.  Rigorously  defined  models  can  directly  support  V&V,  and  this  could  both  subsume  cost 
and  risks.  This  use  case  can  likely  identify  candidate  requirements  for  AVCE.  Therefore,  we 
added  this  use  case  as  a  place  holder,  and  are  considering  a  potential  task  that  relates  to  both 
UC05  and  UC03.  However,  there  are  a  number  of  possible  contribution  to  various  types  of  V&V. 
For  example,  the  effort  to  use  SERC  RT-176  effort  of  Monterey  Phoenix  for  V&V  of 
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requirements  may  support  some  of  this  effort.  The  model  created  by  Georgia  Tech  for  RT-170 
has  other  examples  illustrating  some  V&V.  If  we  are  able  to  use  the  IMCE  ontologies  for  systems 
engineering  from  NASA/JPL,  then  this  would  provide  another  avenue  to  support  V&V. 

11  Access  as  Chief  Engineering  Role  (UC08) 


This  use  case  is  created  so  that  one  of  our  researchers,  experienced  in  systems  engineering  can 
provide  some  level  of  assessment  of  our  overarching  approach  and  contribute  to  the 
requirements  for  AVCE.  We  too  want  to  bring  as  many  technologies  as  possible  into  our  lab  at 
Stevens  in  order  to  assess  the  gaps,  but  are  also  interesting  in  bring  in  Masters  students  to  use 
methods  derived  from  this  research. 

This  use  cases  focuses  on  the  requirements  and  assessment  for  integrating  the  tools  and 
methodologies  that  will  constitute  the  integrated  modeling  environment  based  on  lolF, 
OpenMBEE,  and  other  supporting  tools  as  discussed  in  Section  12.  Specifically,  the  effort 
focuses  on  the  Systems  Engineering  and  creation  of  a  "Minimum  Viable  Demonstrator"  at  the 
Stevens.  We  have  now  assembled  a  lab  with  server  machines  that  are  being  populated  with 
tools  and  examples.  We  are  also  planning  to  investigate  the  use  of  UC03  using  MDAO  methods 
and  tools  in  a  Stevens  course  in  the  Fall  of  2017.  Another  possibility  we  are  considering  for 
phase  two  as  a  stretch  goal,  is  to  recruit  a  student  team  to  do  their  two-semester  capstone 
design  project  (aka  Senior  Design)  with  the  goal  of  participating  in  one  of  the  many  UAS  related 
design  competitions  such  as  http://www.auvsi-suas.org. 

12  Tradeoff  Analysis  of  Technologies  for  Integration  or  Interoperability  (UC09) 


This  use  case  seeks  to  support  the  requirements  analysis  for  AVCE  iMBE  and  demonstrate  new 
concepts  for  using  interoperability  to  achieve  tool-to-tool  integration.  Specifically,  we  are 
looking  at  the  technologies  and  tools  used  by  ARDEC,  as  well  as  other  organizations  who  are 
creating  and  evolving  their  integrated  modeling  environments.  We  have  a  laboratory  to  support 
research  on  the  tradeoff  analysis  of  technologies  for  integration  or  interoperability  in  order  to 
further  study  the  technologies  and  provide  demonstrations.  Most  importantly,  the  lolF 
framework  is  evolving,  and  we  have  provided  several  demonstrations  for  both  integration  and 
interoperability  through  SWT  (UCOO,  UC01,  UC02,  and  UC04). 

This  tasks  revisits  some  of  the  most  advanced  tool  integrations  that  have  been  developed  by 
NASA/JPL  [59]  [10],  the  DARPA  META  projects  [8]  [7],  Engineered  Resilient  Systems  [81],  Airbus 
[76],  and  generalization  of  commercial  and  industry  integrated  modeling  environments. 

We  have  joined  Open  Collaboration  Group  for  MBSE  and  OpenMBEE  [132]  and  look  to  take 
advantages  of  the  OpenMBEE  open  source  tools.  Jeff  McDonald  performed  PTC  Windchill  [176] 
analysis  for  the  Army  under  the  SERC  RT-152  [106].  We  expanded  on  the  Windchill  research  in 
support  of  identifying  capabilities  for  the  AVCE  iMBE  concept.  We  recently  learned  of  Syndeia 
by  Intercax  [167],  attended  a  demonstration  on  March  7th,  2017  with  our  ARDEC  sponsor.  We 
will  look  for  another  plan  to  continue  this  research  on  Syndeia. 
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12.1  Analyzing  Tool  Integrations 

We  initially  started  this  task  looking  at  the  application  of  a  multitude  of  tools  used  in  modern 
product  development,  aligning  mostly  with  MCE.  Complexity  arises  as  the  volume  of  the 
needed  tool  set  and  their  inter-dependencies  increase.  The  design  structure  matrix  (DSM)  has 
been  demonstrated  to  be  very  helpful  for  representing  and  analyzing  the  architecture  of  an 
individual  system,  such  as  a  product,  a  process,  and  an  organization  [51].  A  DSM  is  often  a  two- 
dimensional  matrix  representation  of  the  structural  or  functional  interrelationships  of  objects, 
tasks  or  teams.  Synonyms  for  DSM  can  be  N2-Diagram  ("N-squared"),  and  Dependency 
Structure  Matrix.  Types  of  DSM  found  in  use  include  object-based,  team-based,  parameter- 
based,  task-based,  software  module-based,  and  tool-based. 

In  this  use  case,  we  initially  planned  to  explore  the  potential  of  DSM  in  addressing  challenges 
associated  with  integrating  various  tools  in  product  development.  However,  our  researcher  did 
not  have  detailed  insights  into  many  of  these  tools,  several  of  which  have  been  created  by 
ARDEC  to  serve  very  special  purposes  in  their  analysis  and  designs.  Rich  Swanson  in  the  second 
working  session  discussed  some  of  these  integrations,  but  we  are  not  including  those  details  in 
this  report  due  to  the  labeling  on  the  presentation  material;  we  are  not  distributing  this 
material  either.  Therefore,  we  have  concluded  that  in  order  to  attempt  to  do  the  DSM  analysis, 
we  would  have  need  significant  support  from  ARDEC  or  other  experts  that  can  discuss  how  they 
use  the  tools.  Therefore,  this  section  describes  why  and  how  we  would  attempt  to  perform  this 
type  of  analysis. 


12.2  The  Dynamic  Nature  of  Tool  Integration 

Tool  integrations  are  dynamic  consequences  of  customer  requirements.  Tool  integration  are 
not  simply  statically  putting  a  certain  set  of  tools  together.  Depending  on  the  varying  needs  of 
tasks  from  particular  stakeholders,  the  types  of  tools  needed,  their  execution  sequences,  the 
interdependencies  of  data  flow  among  them  vary  from  case  to  case.  In  addition,  the  problem 
often  gets  worse  when  attempting  to  maintain  an  integration  for  different  versions  of  tools. 
Figure  51  illustrates  the  dynamic  nature  of  tool  integration  [157]. 
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Figure  51.  Coordination  Across  Tools  Based  on  User  Story 

Given  a  particular  user  story,  i.e.  a  requirement,  the  set  of  tools  needed  to  be  integrated, 
together  with  the  interwoven  relations  among  them  should  be  computed,  represented,  and 
analyzed  separately. 


12.2.1  The  Overall  DSM  for  Tool  Integration 

Given  a  comprehensive  set  of  available  tools  that  may  be  potentially  used  in  different  phases  of 
product  development.  We  can  construct  a  DSM  to  represent  their  relationships.  As  a  toy 
example  shown  in  Figure  52,  the  rows  and  columns  can  represent  available  tools,  ranked  in 
layers  following  the  temporal  order  that  tools  can  be  used  in  various  phases  of  product 
development.  Each  cell  in  the  matrix  can  represent  the  dependency  between  the  tool  on  the 
row  and  the  tool  on  the  column.  For  example,  CREO  (a  3D  CAD  software)  may  use  the  design 
blueprint  created  by  the  Prodas  tool  (weapon  design  tool)  for  3D  visualization,  hence,  there 
exist  a  dependency  from  the  CREO  to  Prodas. 
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Figure  52.  Overall  DSM  for  Tool  Integration 


In  general,  the  dependencies  among  tools  form  a  hierarchy,  where  the  later  phase  tools  depend 
on  the  prior  phase  tools.  However,  there  are  exceptions,  where  the  design  and  simulation 
phase  tools  can  depend  on  the  review  phase  tools.  This  is  because  the  review  phase  tools  can 
generate  feedback  information,  which  can  lead  the  product  development  life  cycle  to  iterate 
back  to  re-design  and  re-simulation. 


12.2.2  Splitting  out  a  sub  DSM  for  a  User  Story 

For  a  particular  user  requirement,  i.e.  a  user  story,  a  sub  DSM  can  be  split  out  from  the  overall 
DSM  to  represent  tool  integration  pertinent  to  the  task  at  hand.  This  sub  DSM  focuses  on  the 
necessary  tools  and  their  relationship  relevant  to  the  particular  user  story.  For  example,  for  a 
task  "X"  in  a  certain  problem  domain,  a  sub  DSM  shown  in  Figure  53  can  be  split  out  from 
Figure  52  for  a  more  focused  view. 
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Figure  53.  Tool  Integration  Sub-DSM  for  a  User  Story 


In  summary,  using  the  DSM  representation,  we  can  represent:  1)  the  comprehensive  inter¬ 
dependencies  among  available  tools,  and  2)  the  dynamic  integration  of  any  subset  of  tools  for  a 
particular  task.  Table  2  provides  a  list  of  some  of  the  more  than  80  tools  that  are  considered  for 
integration  or  interoperability. 


Table  2.  ARDEC  Tools  List 


Tool  Name 

Description 

IBM  Rational  DOORS 

Requirements  management  application 

Magic  Draw 

Business  process,  architecture,  software  and  system  modeling  tool  with 
teamwork  support  MBSE.  Also  has  integration  with  ModelCenter  and 
OpenMBEE. 
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AAMODAT 

Excel-based  spreadsheet  tool  that  supports  the  decision  framework  concept 
as  discussed  in  Section  9 

ERS 

NA 

IMPRINT 

NA 

D/S  ABAQUS/CFD 

Provides  advanced  computational  fluid  dynamics  capabilities  with  extensive 
support  for  preprocessing  and  post  processing  provided  in  Abaqus/CAE. 

ANSYS  FLUENT 

ANSYS  Fluent  is  the  most-powerful  computational  fluid  dynamics  (CFD) 
software  tool  available,  empowering  you  to  go  further  and  faster  as  you 
optimize  your  product's  performance. 

ANSYS  FEA/CFD 

NA 

ANSYS  Ansoft  (EE/RF) 

Design  flow  that  for  modeling  and  simulate  complex  analog,  RF,  and  mixed- 
signal  applications  and  perform  signal-integrity  analysis  and  system 
verification  of  high-performance  IC/package/board  designs. 

LabVIEW 

LabVIEW  is  an  integrated  development  environment  designed  specifically  for 
engineers  and  scientists.  Native  to  LabVIEW  is  a  graphical  programming 
language  (G)  that  uses  a  dataflow  model  instead  of  sequential  lines  of  text 
code,  empowering  you  to  write  functional  code  using  a  visual  layout  that 
resembles  your  thought  process. 

MSC  Suite 

NA 

LMS  Virtual  Lab 

an  integrated  suite  of  3D  FE  and  multi-body  simulation  software  which 
simulates  and  optimizes  the  performance  of  mechanical  systems  for 
structural  integrity,  noise  and  vibration,  system  dynamics  and  durability. 

MSTFS 

The  collaboration  platform  at  the  core  of  Microsoft's  application  lifecycle 
management  solution.  TFS  automates  the  software  delivery  process  and  gives 
you  the  tools  you  need  to  effectively  manage  software  development  projects 
throughout  the  IT  lifecycle 

Mathworks 

Matlab,  Simulink,  Stateflow 

Prediction  Probe 

Data  prediction 

JMP  PRO 

Predictive  modeling  and  cross-validation  techniques. 

CALCE 

A  CALCE  methodology  that  uses  physics-of-failure  based  principles  and 
software  to  assess  whether  a  part/system  can  meet  defined  life  cycle 
requirements  based  on  its  materials,  geometry,  and  operating  characteristics. 
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Sherlock  Automated 
Design  Analysis 

A  software  tool  developed  by  DfR  Solutions[l][2]  for  analyzing,  grading,  and 
certifying  the  expected  reliability  of  products  at  the  circuit  card  assembly 
level. 

Erosion 

NA 

Prodas 

Weapon  design  tool 

AutoDesk 

An  American  multinational  software  corporation  that  makes  software  for  the 
architecture,  engineering,  construction,  manufacturing,  media,  and 
entertainment  industries. 

CREO 

3D  CAD  Software 

*NA  means  we  didn't  find  related  information 


12.2.3  Capturing  Workflow  Information  Using  Design  Structure  Matrix 

ARDEC  has  identified  about  85  tools  that  should  be  considered  as  part  of  various  workflows, 
which  cover  the  entire  lifecycle.  As  shown  in  Figure  54,  they  are  investigating  the  use  of  the 
DSM  concept  for  capturing  information  about  the  numerous  workflows  that  exist  at  ARDEC. 

■  Basic  question:  what  tools  provide  information  used  by  other  tools? 

■  Upper/right  portion  (Green)  -  identify  sequence  from  left  to  right. 

■  Lower/Left  portion  (Red)  -  Identify  sequence  from  right  to  left. 

■  Example.  Output  from  Prodas  is  used  as  input  to  CFD  Muzzle  Analysis. 
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Figure  54.  Example:  Output  from  Terminal/Systems  Effects  is  used  as  input  to  CASRED. 


12.3  Canonical  Reference  architecture  of  an  Integrated  MCE  Environment 

Recalling  that  a  critical  element  of  the  first  year  of  this  research  is  to  understand  the 
requirements  for  AVCE  iMBE,  we  believe  that  the  RT-141  final  report  [22]  generalized 
capabilities  heard  by  many  organizations  [7]  [8]  [10]  [43]  [59]  [81]  [90]  [139]  and  characterizes  a 
canonical  reference  architecture  of  an  Integrated  MCE  Environment,  as  shown  in  Figure  55.  The 
following  sub-sections  discuss  various  elements  from  the  canonical  reference  architecture  for 
an  integrated  MCE  environment.  The  following  provides  some  perspectives  and  capabilities  of 
this  vision  concept: 

■  Provides  appropriate  views  for  the  various  stakeholder 

■  Stakeholders  have  views  into  the  Single  Source  of  Truth  (SST) 

■  Using  rich  modeling  interfaces  for  those  with  expertise  in  modeling 
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■  Using  rich  "web"  interface,  which  today  provides  support  for  graphics,  integrated  with 
structure  inputs,  generated  textual  views  and  3D  model  viewing  [144] 

■  MDAO  layer  provides  for  problem  and  design  space  exploration  of 
o  Physics-based  models 

o  Integrity-based  models 
o  Cost  and  scheduling  models 
o  Risk  models 
o  Various  "illities"  models 
o  Including  surrogates  and  components 

■  Enabled  by  High  Performance  Computing  (HPC) 

■  Semantically  rich  linkages  between  data  and  information  in  the  SST  provides  for 
continuous  workflow  orchestration  -  enabled  by  HPC 

■  Document  generation  is  enabled  by 

o  Semantically  rich  links  to  information  in  the  SST 
o  Templates  that  formalize  patterns  for  requirements,  contracts,  etc. 

■  Enabling  technologies  such  as  machine  learning  provides  a  virtual  knowledge  librarian 
that  assist  users  guided  by  embedding  knowledge  and  training 

■  Contractor  and  collaborators  have  a  secure  means  to  plugin  to  view  or  share  digital 
information  as  a  new  paradigm  for  interactions 

■  This  view  of  the  Designing  System  provides  links  downstream  to  fully  link  Product 
Lifecycle  Management  (PLM) 
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Figure  55.  Integrated  Environment  for  Iterative  Tradespace  Analysis  of  Problem  and  Design  Space 

Therefore,  the  elaboration  of  the  subtasks  as  described  in  Section  12  come  from  insights  gained 
in  discussions  from  over  thirty  organizations,  related  SERC  analyses,  and  new  research  findings. 


12.4  Windchill  Analysis 

ARDEC  is  interested  in  how  Windchill  can  support  the  AVCE.  The  RT-152  [106]  technical  report 
indicates  that  PTC  Windchill  [176]  is  a  capable  engineering  design  data  management  tool,  but  it 
has  shortcomings  in  terms  of  integration  with  simulation  tools  and  lifecycle  data  management. 
Additionally,  Windchill  fails  to  provide  the  "real-time"  linkages  to  allow  data  comparison  and 
migration  amongst  various  toolsets.  The  RT-152  report  provides  the  following  summary  (non- 
exhaustive): 

■  Windchill  implementation  requires  a  detailed  plan  that  includes  architecture,  work 
process  revisions,  testing,  training,  and  deployment  requirements 

■  Windchill  should  only  be  configured 

■  Avoid  customizations  to  minimize  impact  on  data  migration,  support  costs,  use  of  third 
party  software,  and  ability  to  upgrade  versions 

■  Windchill  is  an  engineering  tool  that  is  used  by  non-engineers 

■  Windchill  is  not  user  friendly  and  detailed  training  is  required 

■  Training  should  be  tailored  to  each  class  of  users 

This  task  has  additional  research  objectives  and  questions  (non-exhaustive): 

■  Can  the  current  capabilities  of  Windchill  support  the  AVCE  vision? 
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■  How  will  Windchill  data  support  the  concept  of  the  underlying  information  model? 

■  What  is  the  interoperability  of  Windchill  with  other  systems  to  support  the  concept  of 
Single  Source  of  Truth  (STT)? 

■  What  are  the  pros  and  cons  of  Windchill  vs.  the  concept  of  domain  ontologies  using 
SWT? 

■  What  are  the  pros  and  cons  of  Windchill  vs.  other  potential  COTS  solutions  (i.e.  Syndeia) 
for  achieving  the  AVCE  vision? 

Preliminary  analysis  to  these  questions  suggests  that: 

■  Windchill  can  address  some  aspect  of  the  AVCE  Vision 

■  Windchill  is  a  powerful  PDM  tool  that  provides  the  backbone  for  PLM 

■  Windchill  integrates  many  design  tools  (mechanical  CAD,  electrical  CAD,  enterprise 
resource  planning  [ERP],  and  MS  Office) 

o  The  user  defined  relationships  allow  linking  artifacts  to  create  an  integrated 
database 

■  Windchill  has  partnered  with  many  leading  software  designers  to  offer  adaptors  to  link 
data  to  third  party  software  (IBM  Doors,  Solidworks,  ThingWorx) 

■  If  no  partnership  exists,  Windchill  supports  importing  and  exporting  of  engineering  data 
in  multiple  formats  to  support  use  in  third  party  or  custom  software 

o  This  relationship  is  not  linked  to  source  data 

■  Windchill  has  complicated  user  interface  that  requires  extensive  training 

o  One  user  termed  it,  "An  engineering  tool  that  must  be  used  by  non-engineers." 

■  Windchill  cannot  achieve  SST  as  a  stand-alone  product 

■  PTC  offers  complimentary  software  that  when  combine  may  support  a  SST  within  the 
Windchill  environment  (e.g.,  PTC  Integrity  Modeler,  PTC  Windchill  Project  Link,  PTC 
Windchill  Parts  Link) 

■  PTC  Navigate  (compatible  with  Windchill  vlO.l  and  later)  offers  a  user  friendly  html 
based  interface  for  viewing  and  accessing  part  and  document  data  stored  within 
Windchill 

■  PTC  has  recently  developed  multiple  partnerships  to  leverage  advances  in  Internet  of 
Things  (loT)  software  to  integrate  disparate  data  sources 

Our  early  assessment  suggests  that  Windchill  can  provide  support  of  the  underlying  information 
model: 


■  Windchill  currently  integrates  the  product  information  from  multiple  software  tools  and 
can  export  this  data  in  its  native  form  or  as  metadata 

■  Windchill  could  potentially  be  used  as  one  of  the  main  sources  for  data/metadata  for 
the  information  model 

o  Non-integrated  software  data  could  be  fed  to  the  information  model  separately 

■  A  hybrid  solution  could  use  Windchill  as  a  software  source  for  the  data  acquisition  and 
aggregation  layer  to  support  the  High  Level  Framework  Concept 

Table  3  provides  a  summary  of  the  current  analysis: 
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Table  3.  Comparison  of  Approaches  Related  to  Windchill 


Windchill 

PTC  Software  Suite 

Ontologies 

Syndeia 

Achieves  SST 

Good 

? 

? 

Supports  Information  Model 

Excellent 

Excellent 

? 

? 

Supports  Decision  Framework 

? 

? 

? 

? 

Tool  Integration 

Good 

Excellent 

? 

? 

User  Interface 

Good 

? 

? 

Expertise  to  Setup/  Maintain 

Good 

Fair 

? 

? 

Commercial  Availability 

Excellent 

Excellent 

Good 

Excellent 

Good 

? 

Fair 

Poor 

Under  Evaluation 

This  section  also  includes  a  description  of  Syndeia  in  Section  12.5,  which  has  relationships  to 
Windchill  and  other  tool  integrations.  Section  12.7  discusses  another  example  using  the  Airbus 
Space's  Digital  Environment  that  includes  Windchill.  We  will  look  at  other  approaches  such  as 
data  interoperability,  and  specifically  investigating  if  Windchill  can  support  the  SWT  approach 
or  can  operate  using  a  publish/subscribe  approach  flowing  data  to  Windchill  for  the  information 
model  via  proxies  as  discussed  in  Section  12.8. 


12.5  Syndeia 

Syndeia  is  a  software  platform  for  MCE.  It  seeks  to  enable  engineering  teams  to  collaboratively 
develop  and  manage  a  system  model.  It  provides  a  means  to  combine  a  system  architecture 
model  defined  in  languages  such  as  SysML  with  models  in  other  MBE  domain,  including  PLM 
(e.g.  Teamcenter,  Windchill),  CAD  (e.g.  NX,  Creo),  Application  Lifecycle  Management  (ALM)  (e.g. 
GitHub),  Project  Management  (e.g.  JIRA),  Requirements  Management  (e.g.  DOORS-NG), 
Simulations  (e.g.  Mathematica  and  MATLAB/Simulink),  Databases  (e.g.  MySQL),  and  other  data 
sources  (e.g.  Excel). 

This  subtask  looks  to  assess  the  comprehensiveness  of  this  approach  in  the  context  of  ARDEC's 
needs,  and  researching  viable  commercial  alternatives  to  a  SWT  approach.  Specifically,  we  are 
looking  into  PTC  software  toolsets  (PTC  Link)  and  Interval  Syndeia.  Early  paper  analysis  on 
Syndeia  suggests  this  may  offer  a  potential  solution  or  partial  solution  based  upon  its  ability  to 
integrate  other  third  party  software.  Specifically,  we  are  interested  if  proprietary  simulation 
tools  can  be  integrated  into  Syndeia.  The  demonstration  of  Syndeia  to  ARDEC  and  the  RT-168 
team  was  held  on  March  7th,  2017.  We  have  requested  academic  licenses  for  further  analysis. 
We  do  know  that  organizations  like  NASA/JPL  are  using  Syndeia  in  the  context  of  OpenMBEE, 
which  is  described  in  Section  12.6. 
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The  concept  of  a  federated  set  of  software  tools  allows  for  repositories  that  are  optimized  for 
the  type  of  data  that  each  specific  tool  can  store  and  the  workflows  that  create  and  manage 
that  type  of  data.  A  tool  like  Syndeia  can  facilitate  the  mining  of  relationships  across  these 
domain  specific  repositories  that  allows  one  to  build  system  level  models  to  facilitate  the  design 
and  analysis  at  the  system  level.  Because  this  can  be  done  in  "real  time,"  with  what  is  going  on 
at  the  engineering  design  level,  it  opens  up  for  new  ways  of  doing  system  engineering  to  better 
link  across  domains.  Instead  of  a  traditional  "top  down"  approach,  which  prescribes  the 
"specification"  of  the  components  due  to  (a  mostly  unreliable)  step  by  step  process  of 
transforming  user  needs  into  lower  level  specifications,  it  now  be  a  much  more  fluent  and 
adaptive  process  of  guidance  and  facilitation  using  existing  assets,  modified  assets  and  new 
assets  as  needed.  Our  research  team  need  to  see  how  this  paper  analysis  aligns  with  the 
realities  of  how  this  type  of  integration  can  support  a  different  operational  paradigm  for 
systems  engineering.  However,  creating  a  laboratory  with  some  of  the  kinds  of  tools  that 
integrate  through  Syndeia  may  be  challenging  in  the  university  environment. 


12.6  OpenMBEE  and  Open  Collaboration  Group  for  MBSE 

We  recently  joined  the  Open  Collaboration  Group  for  MBSE  that  is  providing  support  for 
adopting  and  contributing  to  OpenMBEE  [132].  We  are  planning  to  use  OpenMBEE  in  our  lab. 
OpenMBEE,  as  shown  in  Figure  56,  is  an  open  source  platform  for  modeling  that  utilizes  the 
Model  Management  System  (MMS)  that  can  be  accessed  from  rich  SysML  desktop  clients  like 
MagicDraw,  and  light-weight  web-based  client  like  View  Editor.  It  provides  infrastructure  for 
fine-grained  versioning  (i.e.,  at  the  object  level,  not  the  file  level),  workflow  management,  and 
access  control.  OpenMBEE  facilitates  multi-tool  and  multi-repository  integration  across 
engineering,  computing,  and  management  disciplines.  OpenMBEE  provides  the  core  allowing 
tracking  relations  between  heterogeneous  data  sources  in  a  linked  data  architecture.  System 
models  are  constructed,  queried  and  rendered  following  the  view  and  viewpoint  paradigm. 
OpenMBEE  was  started  by  NASA/JPL,  but  is  open  sourced  and  there  is  growing  community  that 
includes  industry  users  and  contributors  (e.g.,  Boeing,  Lockheed  Martin). 
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Figure  56.  Conceptual  Elements  of  OpenMBEE 

We  will  highlight  a  few  parts  of  an  instantiation  of  OpenMBEE  as  shown  in  Figure  57. 
ModelCenter,  supporting  MDAO  is  part  of  the  environment.  They  formalize  the  System 
Engineering  modeling  methodology  through  model  patterns  [40]  [109]  that  are  captured 
through  ontologies  using  SWT.  The  approach  is  associated  a  SysML-profiled  modeling  tool 
approach  that  not  only  guides  development,  but  provides  model  analysis  to  ensure  compliance 
with  the  patterns  (e.g.,  models  are  well-formed,  consistent,  etc.)  [90].  There  is  a  video  training 
module  [91]  that  provides  details  about  this  concept  and  tooling. 


Report  No.  $ERC-2017-TR-n0 


96 


Date:  August  8,  2017 


Contract  No.  HQ0034-13-D-0004 


Figure  57.  OpenMBEE  Instantiation  (2014)[118] 


They  formalize  at  least  25  modeling  patterns  applicable  to  systems  engineering  in  ontologies 
using  the  standard  Web  Ontology  Language  (OWL)  [179]  to  provide  a  way  of  defining  a  set  of 
concepts  and  properties  applicable  to  the  domain  of  discourse  of  Systems  Engineering  such  as: 
component,  function,  requirement,  and  work  package,  data  properties  like  mass  and  cost,  and 
object  properties  (relationships)  like  performs,  specifies,  and  supplies.  This  provides  for  a 
controlled  vocabulary  and  enforcing  rules  for  well-formedness,  which  permits,  among  other 
things,  interdisciplinary  information  integration,  and  automated  analysis  and  product 
generation.  Because  the  SE  ontologies  are  expressed  in  OWL,  they  are  amenable  to  formal 
validation  (syntactic  and  semantic)  with  formal  reasoning  tools.  The  approach  embedded  in 
SysML  and  the  OWL  ontologies  is  created  by  transformations  from  SysML  models  [90].  Once  a 
model  is  completed  other  transformations  are  performed  to  the  model,  such  as  checking 
properties  of  well-formedness  and  consistency  of  the  model.  They  currently  have  about  60,000 
test  cases  for  checking  these  types  of  properties.  The  approach  is  illustrated  in  several  case 
studies  [109].  Finally,  we  are  also  interested  in  the  approach  for  automatically  generating  a 
specification  from  a  model,  and  will  experiment  with  using  the  MDK  plugin  [111]  with  DocGen 
through  MagicDraw. 


12.7  Digital  Environment  at  Airbus  Space 

We  have  discussed  the  importance  of  an  underlying  information  model  to  enable  the  cross¬ 
domain  integration  of  information  in  a  single  source  of  truth  [22].  Ralf  Hartmann,  the  Vice 
President  of  Enterprise  Digitization  gave  a  technically  detailed  and  highly  relevant  presentation 
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at  the  NASA/JPL  Symposium  and  Workshop  in  Jan  2017  [76].  While  there  were  many  points,  of 
particular  interest  was  a  historical  perspective  on  how  they  have  been  assembling  a  system 
design  engineering  environment  to  cover  the  entire  lifecycle.  The  representation  of  the 
environment  as  shown  in  Figure  58  was  particularly  interesting  as  it  relates  to  the  concept  of  a 
semantically  rich  information;  this  pertains  to  the  box  in  the  middle  call  RangeDB  Data 
Management.  This  is  a  relatively  recent  development  where  they  replaced  a  commercial 
product  with  their  own  infrastructure  functionality  (i.e.,  "secret  sauce")  that  provides  a 
Semantic  Data  Model  for  multi-disciplinary  Integration  as  shown  in  Figure  59.  We  did  discuss 
this  with  a  person  from  Airbus  at  the  event,  and  asked  about  the  strange  name  (i.e.,  RangeDB), 
and  he  said  it  was  "historical."  This  effort  confirms  why  we  believe  SWT  will  play  a  key  role  to 
characterize  the  underlying  information  model  for  both  ARDEC  and  NAVAIR,  and  again  reflects 
positively  on  the  NASA/JPL  use  of  SWT  as  discussed  in  Section  12.6. 


Systems  ’’’  Functional  Mechanical 

Engineering  Engineering  Engineering 


Figure  58.  Airbus  Digital  End-to-End  (System  &  Product)  Engineering 
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Figure  59.  Semantic  Data  Model  for  Multi-Disciplinary  Integration 

Finally,  the  Hartmann  briefing  also  included  an  associated  roadmap  as  shown  in  Figure  60  that 
was  structured  in  two  dimensions: 

■  Technology  clusters 

o  Requirement  engineering  &  V&V 
o  MBSE  and  design 

o  Engineering  data  lifecycle  management 
o  Collaborative  engineering 

■  System  engineering  technology  integration  levels 
o  Data  integration  (just  connecting  data) 

o  Semantic  integration  (identifies  rules  how  to  connect  and  understand  data) 
o  End-to-end  (knowledge  management) 

The  key  reflection  on  this  roadmap  is  acknowledging  the  increased  need  to  formalize  the 
underlying  information  model  as  we  move  to  the  right  (i.e.,  future),  which  can  exploit  more 
computational  automation  enabled  by  high  performance  computing. 
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Figure  60.  Airbus  Roadmap  Shown  Bands  of  Digital  Engineering  Integration 


12.8  RT-168  Tool-to-Tool  Integrating  and  Interoperability  Framework 

Given  the  context  about  integrated  modeling  environments,  we  have  been  able  to  assemble 
some  relevant  tools.  Our  researcher  Roger  Blake  also  runs  a  laboratory  and  can  provide  the 
resources  to  experiment  with  both  tool-to-tool  integration,  as  well  as  operational  models.  This 
section  discusses  the  evolving  state  of  those  integrations  by  providing  an  overview  of  Tool-to- 
Tool  Integrating  and  Interoperability  Framework  (lolF). 

The  lolF  under  UC01,  UC02,  UC3  and  UC04  provides  some  software  tool(s)  and  data  acquisition 
functionality,  but  we  will  need  to  coordinate  the  ideas  of  what  their  software  tools  are 
calculating  so  that  we  have  consistency  from  the  data  output  of  the  software  tools  and  into  the 
VR-Forces  Simulation  Model  [103].  This  framework  is  being  designed  to  be  used  with  various 
software  tools  and  various  simulation  environments.  As  reflected  in  Figure  61,  the  immediate 
goals  are: 

■  Abstracts  away  from  the  software  client  tools  as  much  knowledge  and  dependencies  of 
the  tool-to-tool  data  integration  architecture  as  possible 

■  Allows  for  tool-to-tool  data  integration  on  computer  systems  that  are  physically  remote 
from  each  other 

■  Uses  an  ontology  framework  (i.e.,  SWT)  that  implements  an  automated  decision  process 
regarding  tool/data  relationships 
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■  Uses  a  Publish  /  Subscribe  framework  that  implements  an  automated  data  transport 
layer  between  various  software  client  tools 

■  Creates  a  storyboard  regarding  the  prototypes  purpose 

■  We  need  to  understand  how  we  can  leverage  OpenMBEE 


Figure  61.  RT-168  Tool-to-Tool  Integration  and  Interoperability  Framework 


13  Research  Semantic  Web  Technologies  applied  to  AAMODAT  (UC10) 


This  use  case  relates  to  UCOO  and  UC06,  and  the  new  challenge  area  #5.  The  plan  is  to  develop 
an  initial  ontology  that  will  demonstrate  the  ability  of  ontology  driven  SWT  to  parse,  infer,  and 
restructure  data  for  input  into  the  AAMODAT  excel  file.  The  sUAV  case  developed  my  Matt  Cilli 
created  for  the  Engineering  Resilient  System  (ERS)  research  may  work  well  for  the  ontology 
demonstration.  To  develop  an  ontology,  we  need  to  understand  the  data  that  we  need  to  parse 
(documents,  data  bases,  standards,  etc.)  and  then  we  need  to  understand  how  we  need  to  put 
it  back  together  (restructure)  it  for  AAMODAT.  These  elements  would  include: 

■  Objective  hierarchies 

■  Value  functions 

■  Assessment  Flow  Diagrams  (AFDs)  trace  the  relationships  between  physical  means, 
intermediate  measures,  and  fundamental  objectives 

■  Uncertainties 

The  demonstration  and  discussion  at  the  third  working  session  covered  how  AAMODAT  is 
usually  something  that  happens  early  on  for  ARDEC,  and  all  over  the  project.  It  has  helped  to 
identify  Key  Performance  Parameters  (KPPs)  at  the  mission  level  and  the  elements  from  the 
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sub-domains  that  are  relevant  to  those  KPPs.  'All  requirements  are  tradeable/  but  looking  at 
how  much  they  contribute  to  the  KPPs,  is  a  different  way  of  thinking. 

In  the  "old"  process  of  AAMODAT  -  with  the  given  objectives,  Measure  of  Objectives,  etc.  we 
had  to  go  to  SMEs  to  populate  data  in  AAMODAT.  SMEs  would  look  at  historical  data  and  tools 
to  provide  the  information.  Matt  Cilli  believes  most  of  this  can  be  automated,  but  still  some 
SME  augmentation  is  required  to  sign  off  or  to  choose  an  option  (i.e.,  identifying  the  objective 
functions)  as  illustrated  by  Mary's  demonstration. 

The  demonstration  using  SWT  and  DBpedia  provided  good  support  for  this  concept.  DBpedia  is 
a  crowd-sourced  community  effort  to  extract  structured  information  from  Wikipedia,  which 
make  this  information  available  on  the  Web  in  a  SWT-compliant  manner.  Mary  used  a  DBpedia 
database  populated  with  aircraft  data.  DBpedia  employs  its  ontology  to  go  to  Wikipedia  (that 
has  both  structured  and  unstructured  data),  grabs  data  and  bring  it  in  to  DBpedia  as  RDF  data 
(base  data  format  for  SWT).  Turtle,  OWL  are  RDF  formats.  Once  data  is  inside  DBpedia,  it  is 
ready  to  be  queried. 

■  Web  Ontology  Language  (OWL)  can  be  thought  of  as  a  type  schema 

■  Resource  Description  Framework  (RDF)  is  a  standard  model  for  data  interchange  on  the 
Web;  think  about  RDF  as  the  data  elements  that  should  be  compliant  with  an  ontology 
defined  using  OWL 

■  Turtle  (Terse  RDF  Triple  Language)  is  a  format  for  expressing  data  in  RDF 

■  Examples  were  presented  at  the  demo 

The  demonstration  illustrated  concretely  with  visualization  using  Protege,  the  DBpedia 
ontology,  which  is  a  class  structure.  In  DBpedia,  'aircraft'  is  a  subclass  of  'means  of 
transportation'  and  so,  it  inherits  all  properties  of  the  class  above  it.  The  demonstration  used 
the  Protege  tool,  which  is  an  open  source  ontology  editing  tool,  and  DBpedia,  which  does  a  lot 
of  background  checking  to  the  data  that  it  pulls  is  from  Wikipedia. 

14  Assess  AVCE  iMBE  (UC11) 


Mark  Blackburn  was  requested  by  ARDEC  to  provide  a  peer  review  of  the  Requirements  for 
AVCE  iMBE.  The  material  provided  were  traditional  text-based  requirements.  Our  first  major 
comment  was  that  if  we  are  moving  away  from  document-centric  requirements,  then  we 
should  develop  a  model  for  such  requirement,  much  like  the  OpenMBEE  model.  Mark  during 
the  review  added  some  packages  to  the  RT-168  MagicDraw  SysML  model  and  started  adding 
use  cases  implied  by  the  textual  requirement  statement.  We  also  added  other  use  cases  and 
some  associated  relationships  derived  from  our  knowledge  of  those  environments,  including 
OpenMBEE.  We  supplied  the  model  to  ARDEC,  and  have  received  their  model  of  requirements 
for  AVCE  iMBE,  but  have  not  had  an  opportunity  to  thoroughly  review  the  model. 

While  ARDEC  has  finished  the  SRR  for  AVCE  iMBE,  we  asked  Rick  Dove  to  join  RT-168  research 
team,  because  Rick  has  done  some  interesting  work  on  the  INCOSE's  Agile  Systems  Engineering 
Life  Cycle  Model  (ASELCM)  project,  and  specifically,  the  ASELCM  Pattern  of  Three  Concurrent 
Systems.  Agile  systems  engineering  encompasses  three  nested  concurrent  systems,  depicted  in 
Figure  62  as  an  iconic  pattern.  The  pattern  is  the  work  of  Bill  Schindel,  a  principle  co-author  in 
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the  ASELCM  case  studies.  The  ASELCM  Pattern  establishes  a  set  of  system  reference 
boundaries.  Whether  the  systems  of  interest  are  small  or  large,  human  or  inanimate,  flying 
through  the  air  or  performing  business  processes. 


Figure  62.  Notional  Relationships  of  Systems  1,  2,  and  3  [154] 

This  ASELCM  Pattern  particularly  refers  to  three  major  system  reference  boundaries,  and  within 
those,  six  subsystem  reference  boundaries.  These  are  all  logical  boundaries  (defined  by  the 
behavior,  not  the  identity,  of  systems): 

■  System  1:  The  Target  System,  the  subject  of  innovation  over  managed  life  cycles  of 
development,  deployment,  and  support. 

o  Normally,  one  would  think  about  the  target  system  as  the  one  that  ARDEC  would 
deploy  (e.g.,  fire  control,  munitions) 
o  In  this  case,  however,  the  target  system  is  AVCE  iMBE 

■  System  2:  The  Target  System  Life  Cycle  Domain  System,  including  the  entire  external 
environment  of  the  Target  System— everything  with  which  it  directly  interacts, 
particularly  its  operational  environment  and  all  systems  that  manage  the  life  cycle  of  the 
Target  System.  This  includes  the  external  environment  of  the  operational  target 
system(s),  as  well  as  all  the  (agile  or  other)  development,  production,  deployment, 
support,  security,  accounting,  performance,  and  configuration  management  systems 
that  manage  System  1. 

■  System  3:  The  System  of  Innovation,  which  includes  System  1  and  2  along  with  the 
systems  managing  (improving,  deploying,  supporting)  the  life  cycle  of  System  2.  This 
includes  the  systems  that  define,  observe,  analyze  (as  in  agile  software  process 
retrospective),  improve  and  support  processes  of  development,  deployment,  service,  or 
other  managers  of  System  1.  System  1  is  contained  in  System  2,  which  is  contained  in 
System  3.  All  are  (or  at  least  should  be)  happening  simultaneously,  effectively  an  organic 
complex  system  motivated  by  self-preservation  to  evolve  suitably  in  an  uncontrolled 
operational  environment.  Think  of  the  arrow-pointed  pipes  of  Figure  62  as  a  circulatory 

system. 
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15  SERC  Research  Synergies 


An  early  request  of  ARDEC  was  for  our  research  team  to  help  them  increase  awareness  and 
synergies  with  other  organizations.  This  section  discusses  some  synergies  to  the  ongoing  ARDEC 
research  tasks  that  are  briefly  mentioned  in  this  report  to  inform  readers  of  the  relationships  to 
these  other  activities. 


15.1  RT-170  NAVAIR  Systems  Engineering  Transformation  Through  Model  Centric  Engineering 

There  are  many  related  research  efforts  between  ARDEC  and  NAVAIR,  as  well  as  other 
government  organization  that  are  working  towards  and  SE  transformation  using  MCE.  However, 
the  domains  and  concern  are  different  way,  therefore,  we  are  working  with  different  and 
complementary  researchers  to  cross-pollinate  the  results.  This  includes: 

■  Strategies  related  to  MBSE  supported  by  our  Georgia  Tech  collaborators  (Dr.  Russell 
Peak,  Steven  Edwards) 

■  Approaches  to  use  SWT  investigating  cross-domain  integration,  requirements 
ontologies,  Natural  Language  Processing  of  requirements,  supported  by  Mary  Bone  and 
our  University  of  Maryland  collaborators  (Dr.  Mark  Austin,  Dr.  Leonard  Petgna) 

■  MDAO  examples  of  UAVs 


15.2  RT-176  Verification  and  Validation  (V&V)  of  System  Behavior  Specifications 

Our  NAVAIR  sponsor  had  requested  that  the  SERC  RT-176  research  task  being  led  by  Dr.  Kristin 
Giammarco  be  aligned  with  the  ongoing  research  from  RT-157  and  RT-170.  RT-176  aims  to 
leverage  and  extend  existing  research  in  the  area  of  methods,  processes  and  tools  for 
performing  early  Verification  &  Validation  (V&V)  of  requirements  and  architecture  models 
managed  within  its  organization,  and  to  educate  its  workforce  in  the  use  of  automated  tools  for 
conducting  early  and  continuous  V&V  across  the  entire  lifecycle.  We  have  shared  our  UAV 
system  model  and  hope  that  this  model  will  be  developed  as  a  surrogate  to  actual  systems 
under  development  at  NAVAIR  for  use  as  a  case  study  to  test  new  or  improved  methods, 
processes  and  tools  that  are  developed  based  on  those  summarized  in  the  background  and  as  a 
result  of  this  task,  which  are  expected  to  apply  to  other  systems  in  many  domains  throughout 
DoD. 


15.3  Aerospace  Industry  Association  CONOPS  for  MBSE  Collaboration 

This  is  a  follow-up  to  the  effort  completed  last  year  which  developed  a  white  paper  on  the  Life 
Cycle  Benefits  of  Collaborative  MBSE  Use  for  Early  Requirements  Development^].  This  white 
paper  discusses  the  current  state  and  benefits  of  MBSE  across  the  entire  life  cycle  and  provides 
proposals  for  addressing  such  issues  as  MBSE  Collaborative  Framework,  Government  Data 
Rights,  Intellectual  Property,  and  Life  Cycle  Effectiveness  with  MBSE. 
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The  effort  for  this  year  involves  many  of  the  industry  contractors  to  NAVAIR  and  DoD.  The 
results  should  produce  a  white  paper  describing  a  CONOPS  for  how  industry  and  government 
can  collaborate  through  MCE/MBSE. 


15.4  OpenMBEE  and  Open  Collaboration  Group  for  MBSE 

We  recently  joined  the  Open  Collaboration  Group  for  MBSE  that  is  providing  support  for 
adopting  and  contributing  to  OpenMBEE  [132].  We  are  planning  to  use  OpenMBEE  in  our  lab, 
and  contribute  to  the  community  effort  in  order  to  advance  it  with  capabilities  developed 
under  RT-168,  RT-170  and  RT-176. 


15.5  Semantic  Technologies  Foundation  Initiative  for  Systems  Engineering 

The  NASA/JPL  Symposium  and  Workshop  on  MBSE  had  a  keynote  talk  given  by  Steve  Jenkins 
that  was  fundamentally  based  on  SWT  and  a  foundational  ontology  for  Systems  Engineering  as 
discussed  in  Section  3.1.  There  were  also  two  breakout  session  on  the  subject  SWT.  There  was 
significant  attendance  at  the  break  out  session  title:  "Ontologies,  Formalisms,  &  Reasoning" 
possibly  due  to  the  motivation  given  by  Steve  Jenkins.  In  general,  there  is  progress  being  made 
in  this  area  and  there  is  significant  interest.  Dinesh  Verma  has  initiated  an  effort  with  the 
support  of  Steve  Jenkins  and  Mark  Blackburn  to  bring  a  community  of  people  together  in  an 
attempt  to  create  and  ecosystem  on  Semantic  Technologies. 

The  working  group  has  created  a  charter  and  mission: 

■  Charter 

o  The  Semantic  Technologies  Foundation  Initiative  for  Systems  Engineering  is  to 

promote  and  champion  the  development  and  utilization  of  ontologies  and  semantic 
technologies  to  support  system  engineering  practice,  education,  and  research. 

■  Mission 

o  The  mission  of  the  initiative  is  to  collect  a  suite  of  interoperable  ontologies  that  are 
logically  well-formed  and  accurate  from  both  scientific  and  engineering  points  of 
view.  The  initiative  will  charter  a  collective  of  stakeholders  that  are  committed  to 
collaboration  and  adherence  to  shared  semantic  principles  for  the  advancement  of 
systems  engineering.  To  achieve  this,  initiative  working  group  participants  will 
voluntarily  adhere  to  and  contribute  to  the  development  of  an  evolving  set  of 
principles  including  open  use,  collaborative  development,  and  non-overlapping  and 
appropriately-scoped  content.  They  will  capture  and  maintain  metadata  for  each 
ontology  to  encourage  implementation  and  reuse. 


15.6  Digital  Engineering  Working  Group 

We  are  also  participating  in  the  Digital  Engineering  Working  Group,  in  which  both  NAVAIR  and 
ARDEC  are  participating.  The  Office  of  Deputy  Assistant  Secretary  of  Defense  for  Systems 
Engineering  (ODASD(SE))  formalized  the  goals,  which  are: 
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■  Gl.  Formalize  the  development,  integration  and  use  of  models  to  inform  enterprise  and 
program  decision  making. 

■  G2.  Provide  an  enduring  authoritative  source  of  truth. 

■  G3.  Incorporate  technological  innovation  to  link  digital  models  of  the  actual  system  with 
the  physical  system  in  the  real  world. 

■  G4.  Establish  a  supporting  infrastructure  and  environment  to  perform  activities, 
collaborate  and  communicate  across  stakeholders. 

■  G5.  Transform  a  culture  and  workforce  that  adopts  and  supports  Digital  Engineering 
(DE)  across  the  lifecycle. 

These  goals  are  working  toward  realizing  the  benefits  that  were  found  in  Phase  I  and  identified 
at  a  recent  Government-Industry  DE  forum  conducted  by  the  SERC  and  the  Office  of  the  Deputy 
Assistant  Secretary  of  Defense  for  Systems  Engineering.  The  benefits  of  a  DE  transformation  are 
[46]: 

■  Improved  Acquisition  -  by  accepting  digital  deliverables  could  improve  the  governments 
understanding  of  a  projects  status  and  risk  along  with  allowing  them  to  "validate"  the 
contractor's  deliverables. 

■  Improved  Efficiency  and  Effectiveness  -  reduce  time  and  effort  in  the  performance  of 
existing  tasks  using  a  single  source  of  truth  for  the  system. 

■  Improved  Communication;  Better  Trade-Space  Exploration;  Reduced  Risk  -  using 
ontology-based  information  models  to  translate  and  extract  useful  information  between 
a  variety  of  models  and  model  types  could  allow  for  improved  communication  among 
specialists.  This  enables  the  goal  of  the  DoD  to  establish  a  supporting  infrastructure  and 
environment  to  perform  activities,  collaborate  and  communicate  across  stakeholders. 

■  Improved  Designs  and  resulting  Systems  and  Solutions  -  being  able  to  understand  the 
impact  of  requirement  and/or  design  decisions  early  could  help  improve  the  overall 
system  design  and  identify  adverse  consequences  of  the  design  before  committing  to  a 
design  choice.  This  enables  the  DoD  goal  to  formalize  the  development,  integration  and 
use  of  models  to  inform  enterprise  and  program  decision  making  through  an 
authoritative  source  of  truth. 

The  special  session  on  Systems  Engineering  Transformation  through  Model  Centric 
Engineering  Past-Why,  Present-What,  and  Future-How  held  on  July  31st  at  Stevens  with  our 
ARDEC  and  our  Office  of  the  Deputy  Assistant  Secretary  of  Defense  for  Systems  Engineering 
sponsors,  included  some  other  special  guest  from  Digital  Warfare  Office,  Naval  Surface  Warfare 
Center,  MITRE  and  Raytheon.  We  had  a  breakout  session  looking  at  the  risk  and  priorities 
associated  with  the  mapping  future  research  areas  to  goals  of  digital  engineering 
transformation  strategy  as  shown  in  Figure  63. 
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Future  Research  Areas 

Gl.  Formalize  the 

development,  integration 

and  use  of  models  to 

inform  enterprise  and 

program  decision  making. 

G2.  Provide  an  enduring 

authoritative  source  of 

truth. 

G3.  Incorporate 

technological  innovation  to 

link  digital  models  of  the 

actual  system  with  the 

physical  system  in  the  real 

world. 

G4.  Establish  a  supporting 

infrastructure  and 

environment  to  perform 

activities,  collaborate  and 

communicate  across 

stakeholders. 

G5. Transform  a  culture 

and  workforce  that  adopts 

and  supports  DE  across 

the  lifecycle. 

Cross -discipline  integration  of  models  to  address  the 
heterogeneity  of  the  various  tools  and  environments  using 
semantic  technology 

X 

X 

X 

X 

X 

High  Performance  Computing  (HPC)  advancements 
such  as;  1)  supporting  organizing  and  analyzing  “Big 
Data”  and  2)  being  able  to  program  in  parallel  to  take 
advantage  of  HPC  capabilities,  are  needed  to  support  the 
DE  effort 

X 

X 

X 

X 

Model  integrity  to  ensure  trust  in  the  model  predictions 
by  understanding  and  quantifying  margins  and  uncertainty 

X 

X 

X 

X 

X 

Modeling  methodologies  that  can  embed  demonstrated 
best  practices  and  provide  computational  technologies 
for  real-time  training  within  digital  engineering 
environments 

X 

X 

X 

X 

Model  composability  to  understand  the  possibilities, 
constraints  and  rulesets  for  composition  of  multiple 
models 

X 

X 

Human-model  task  allocation  to  understand  what 
activities  are  best  performed  by  human  decision  makers 
and  what  can  effectively  be  automated  or  augmented  with 
model  intelligence 

X 

Workforce  development  to  understand  what  is  needed 
to  educate  model  developers,  users  and  decision  makers 
to  work  in  a  DE  environment 

X 

MCE  acquisition  to  understand  the  needed  changes  to 
acquisition  and  security  when  developing  in  the  new  DE 
environment 

X 

X 

X 

X 

Figure  63.  Mapping  Future  Research  Areas  to  Digital  Engineering  Transformation  Goals 


15.7  National  Defense  Industry  Association  Modeling  and  Simulation 

National  Defense  Industry  Association  (NDIA)  Modeling  and  Simulation  group  is  looking  at 
approaches  for  using  digital  engineering  for  competitive  down  select.  We  are  involved  in  all  of 
these  efforts  to  further  the  objectives  of  our  sponsor  in  August  of  2016  [120]. 


15.8  Symposia  and  Working  Groups 

The  RT-168  researchers  have  also  attended  a  number  of  events,  not  necessarily  funded  under 
RT-168,  however,  these  events  do  have  relevance  to  informing  our  research,  and  we  have 
delivered  meeting  notes  related  to  these  events  which  include: 

■  NASA/JPL  Symposium  and  Workshop  on  Model  Based  System  Engineering,  January  25- 
27,  2017. 

■  MBSE-related  Events  at  INCOSE  International  Workshop,  January  28-31,  2017. 
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16  Part  II  Summary 


This  final  technical  report  summarizes  the  accomplishments  for  this  Phase  I  research.  The 
report  also  outlines  the  refinement  of  the  tasks  with  a  mapping  to  evolving  use  cases  that 
associate  the  roles  of  the  various  researchers  and  ARDEC  stakeholders  to  other  linked  use  cases 
to  show  a  non-exhaustive  set  of  dependencies.  These  dependencies  reflect  on  cross-domain 
concerns,  where  discipline-specific  stakeholders  will  ultimately  use  different  technologies, 
methods  and  associated  analyses.  We  think  this  collective  set  of  use  cases  that  are  being 
researched  in  the  context  of  various  related  UAV/UAS  operational  scenarios  and  case  studies 
are  helping  us  understand  both  technology  and  socio-technical  concerns  that  can  provide 
inputs  to  operational  scenarios  and  requirements  for  AVCE  iMBE. 

This  report  includes  the  updates  characterizing  demonstrations,  deliverables,  reports  and 
research  analyses  presented  during  the  bi-weekly  status  meeting,  as  well  as  the  information 
presented  at  five  working  sessions,  one  special  event,  and  19  virtual  meetings  that  include,  but 
are  not  limited  to: 

■  Demonstrations  of  concepts,  technologies  and  framework  to  leverage  integration  and 
interoperability  that  provides  computationally  enabled  systems  engineering  to  address 
the  challenges  of  cross-domain  model  integration  of  increasingly  complex  cyber  physical 
systems 

■  Demonstrations  of  mission  and  system -of-system  engineering  analysis  for  new 
operational  approaches  such  as  graphical  CONOPS  through  mission-level,  system-level, 
and  component-level  model-centric  engineering 

■  Bringing  concepts  developed  by  NASA/JPL  OpenMBEE  and  specifically  the  Model 
Development  Kit  (MDK)  DocGen  component,  where  we  have  developed  a  number  of 
View  and  Viewpoint  hierarchies  for  using  DocGen,  including  generation  of  the 
"specification"  for  AVCE  iMBE 

■  Concept  for  integrating  Graphical  CONOPS  gaming  technology  to  expose  functionality, 
interfaces,  controls,  and  parametric  details  that  are  going  to  be  analyzed  using 
Multidisciplinary  Design,  Analysis  and  Optimization  at  the  mission-level 

■  Characterizing  metadata  extracted  from  the  Early  Synthetic  Prototyping  (ESP)  work  to 
inform  the  information  model  that  captures  measures  associated  with  human 
interaction  of  ESP  and  more  generally  concepts  such  as  the  graphical  CONOPS 

■  SWT  application  to  the  Decision  Framework  and  AAMODAT  and  formalization  of  a 
concept  for  using  MDAO  workflows  to  represent  the  Assessment  Flow  Diagrams  in  a 
formal  way  that  could  be  automatically  extracted  from  SysML  models  to  populate  SWT 
for  automating  the  population  of  AAMODAT 

■  Provides  methodological  guidance  for  identifying  Key  Performance  Parameters 

■  Facilitated  several  research  synergies  both  SERC  (e.g.,  NAVAIR  and  non-SERC  (NASA/JPL, 
commercial)  to  increases  ARDEC's  knowledge  and  leverage  insights  and  foster  synergies 
from  other  organizations  we  have  been  able  to  leverage 

■  Facilitate  the  acquisition  and  application  of  "high-end"  MCE  commercial  technologies  to 
ensure  that  the  research  questions  are  posed  in  the  context  of  the  most  advanced 
technologies  used  by  government  and  industry 
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■  Align  ARDEC  and  NAVAIR  research  with  the  DoD  Digital  Engineering  Transformation 
Strategy 

We  are  currently  working  with  our  ARDEC  sponsors  to  define  specific  plans  for  RT-168  Phase  II 
(August  2017  through  August  2018)  that  still  fundamentally  align  with  the  current  set  of  use 
cases,  but  with  more  integration  provided  with  and  through  the  lolF  including  the  latest  SWT, 
and  an  ARDEC-aligned  set  of  ontologies. 
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17  Acronyms  and  Abbreviation 


This  section  provides  a  list  of  some  of  the  terms  used  throughout  the  paper.  The  model  lexicon 
should  have  all  of  these  terms  and  many  others. 

2D  Two  dimensions 


3D 

AADL 

ACAT 

ACES 

AFD 

AFT 

AGI 

AGM 

AGS 

ALM 

AMMODAT 

ANSI 

AP233 

API 

AR 

ARDEC 

ASELCM 

ASR 

ATL 

AVCE 

AVSI 

BDD 

BN 

BNF 

BOM 

BPML 

C-BML 

CAD 

CASE 

CDR 

CEO 

CESUN 

CFD 

CGF 

CMM 

CMMI 

CONOPS 

CORBA 

COTS 

CPS 

CREATE 

Three  dimensions 

Architecture  Analysis  &  Design  Language 

Acquisition  Category 

Automated  Concurrent  Engineering  System 

Assessment  Flow  Diagram 

Architecture  Framework  Tool  of  NASA/JPL 

Analytical  Graphics,  Inc. 

Acquisition  Guidance  Model 

Army  Game  Studio 

Application  Lifecycle  Management 

Armament  Analytics  Multiple  Objective  Decision  Analysis 

American  National  Standards  Institute 

Application  Protocol  233 

Application  Programming  Interface 

Augmented  Reality 

Armament  Research,  Development  and  Engineering  Center 

Agile  Systems  Engineering  Life  Cycle  Model 

Alternative  System  Review 

ATLAS  Transformation  Language 

Armament  Virtual  Collaboratory  Environment 

Aerospace  Vehicle  Systems  Institute 

SysML  Block  Definition  Diagram 

Bayesian  Network 

Backus  Naur  Form 

Bill  of  Material 

Business  Process  Modeling  Language 

Coalition  Battle  Management  Language 

Computer-Aided  Design 

Computer-Aided  Software  Engineering 

Critical  Design  Review 

Chief  Executive  Officer 

International  Engineering  Systems  Symposium 

Computational  Fluid  Dynamic 

Computer  Generated  Forces 

Capability  Maturity  Model 

Capability  Maturity  Model  Integration 

Concept  of  Operations 

Common  Object  Requesting  Broker  Architecture 

Commercial  Off  The  Shelf 

Cyber  Physical  System 
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Environments 
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cUAS 

Counter  UAS 

CWM 

Common  Warehouse  Metamodel 

DAA 

Data  Acquisition  and  Aggregation  layer 

DASD 

Deputy  Assistant  Secretary  of  Defense 

dB 

Decibel 

DBMS 

Database  Management  System 

DAG 

Defense  Acquisition  Guidebook 

DARPA 

Defense  Advanced  Research  Project  Agency 

DAU 

Defense  Acquisition  University 

DCDR 

Digital  design  from  Critical  Design  Review  (CDR) 

DE 

Digital  Engineering 

DIS 

Distributed  Interactive  Simulation 

DL 

Descriptive  Logic 

DLR 

DoD 

Department  of  Defense 

DoDAF 

Department  of  Defense  Architectural  Framework 

DoE 

Design  of  Experiments 

DOORS 

Requirement  Management  product 

DOORS-NG 

DOORS-Next  Generation 

DSEEP 

Distributed  Simulation  Engineering  and  Execution  Process 

DSL 

Domain  Specific  Languages 

DSM 

Domain  Specific  Modeling 

DSM 

Design  Structure  Matrix 

DSML 

Domain  Specific  Modeling  Language 

E/DRAP 

Engineering  Data  Requirements  Agreement  Plan 

ERP 

Enterprise  Resource  Planning 

ESP:HE 

ESP:  Higher  Echelon 

ERS 

Engineered  Resilient  Systems 

ESP 

Early  Synthetic  Prototype 

FAA 

Federal  Aviation  Administration 

FEA 

Finite  Element  Analysis 

FMEA 

Failure  Modes  and  Effects  Analysis 

FMI 

Functional  Mockup  Interface 

FMU 

Functional  Mockup  Unit 

FOM 

Federation  Object  Model 

GAO 

Government  Accounting  Office 

GUI 

Graphical  User  Interface 

HLA 

High  Level  Architecture 

HPC 

High  Performance  Computing 

HPCM 

High  Performance  Computing  Modernization 

HW 

Hardware 

l&l 

Integration  and  Interoperability 

IBM 

International  Business  Machines 

IBD 

Internal  Block  Diagram  (SysML) 

ICD 

Interface  Control  Document 

ICT 

Institute  for  Creative  Technologies 

ICTB 

Integrated  Capability  Technical  Baseline 

IDEFO 

Icam  DEFinition  for  Function  Modeling 
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IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

iMBE 

AVCE-Integrated  Model-Based  Engineering 

INCOSE 

International  Council  on  Systems  Engineering 

IPR 

Integration  Problem  Report 

lolF 

Integration  and  Interoperability  Framework 

IRL 

Integration  Readiness  Level 

ISEDM 

Integrated  Systems  Engineering  Decision  Management 

ISEF 

Integrated  System  Engineering  Framework  developed  by  Army's  TARDEC 

ISO 

International  Organization  for  Standardization 

IT 

Information  Technology 

IWC 

Integrated  Warfighter  Capability 

JCIDS 

Joint  Capabilities  Integration  and  Development  System 

JEO 

Jupiter  Europa  Orbiter  project  at  NASA/JPL 

JSF 

Joint  Strike  Fighter 

JPL 

Jet  Propulsion  Laboratory  (NASA) 

JSON 

JavaScript  Object  Notation 

KPP 

Key  Performance  Parameter 

KSA 

Key  System  Attributes 

LIDAR 

Light  Detection  and  Ranging 

LOC 

Lines  of  Code 

LSL 

Lab  Streaming  Layer 

M&S 

Modeling  and  Simulation 

MARTE 

Modeling  and  Analysis  of  Real  Time  Embedded  systems 

MATRIXx 

Product  family  for  model-based  control  system  design  produced  by  National 
Instruments;  Similar  to  Simulink 

MBE 

Model  Based  Engineering 

MBEE 

Model  Based  Engineering  Environment 

MBSE 

Model  Based  System  Engineering 

MBT 

Model  Based  Testing 

MC/DC 

Modified  Condition/Decision 

MCE 

Model  Centric  engineering 

MDA® 

Model  Driven  Architecture® 

MDAO 

Multidisciplinary  Design,  Analysis  and  Optimization 

MDD™ 

Model  Driven  Development 

MDE 

Model  Driven  Engineering 

MDK 

MagicDraw  Model  Development  Kit 

MDSD 

Model  Driven  Software  Development 

MDSE 

Model  Driven  Software  Engineering 

MIC 

Model  Integrated  Computing 

MMM 

Modeling  Maturity  Model 

MMS 

Model  Management  System  (part  of  OpenMBEE) 

MoDAF 

Ministry  of  Defence  Architectural  Framework  (United  Kingdom) 

MOE 

Measure  of  Effectiveness 

MOF 

Meta  Object  Facility 

MOP 

Measure  of  Performance 

MRL 

Mixed  Reality  Lab 

MxRP 

Mixed  Reality  Prototyping 

MSDL 

Military  Scenario  Definition  Language 
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MVS 

Multiple  Virtual  Storage 

N2 

N-squared  diagram 

NASA 

National  Aeronautics  and  Space  Administration 

NASA/JPL 

NASA  Jet  Propulsion  Laboratory 

NAVAIR 

U.S.  Navy  Naval  Air  Systems  Command 

NAVSEA 

U.S.  Naval  Sea  Systems  Command 

NDA 

Non-disclosure  Agreement 

NDIA 

National  Defense  Industrial  Association 

NEAR 

Naval  Enterprise  Architecture  Repository 

NPS 

Naval  Postgraduate  School 

NSGA 

Non-dominated  Sorting  Genetic  Algorithm 

OCL 

Object  Constraint  Language 

OMG 

Object  Management  Group 

00 

Object  oriented 

OpenMBEE 

Open  Model  Based  Engineering  Environment 

OpenVSP 

Open  Vehicle  Sketch  Pad 

OSD 

Office  of  the  Secretary  of  Defense 

OSLC 

Open  Services  for  Lifecycle  Collaboration 

OV1 

Operational  View  1  -  type  of  DoDAF  diagram 

OWL 

Web  Ontology  Language 

PAR 

Parametric  Block  in  SysML 

PDM 

Product  Data  Management 

PDR 

Preliminary  Design  Review 

PEA 

Post  Exercise  Analysis 

PES 

Physical  Exchange  Specification 

PIA 

Proprietary  Information  Agreement 

PIM 

Platform  Independent  Model 

PLM 

Product  Lifecycle  Management 

POR 

Program  of  Record 

PRR 

Production  Readiness  Review 

PSM 

Platform  Specific  Model 

QMU 

Quantification  of  Margins  and  Uncertainty 

RDEC 

US  Army  Research  Development  and  Engineering  Center 

RDF 

Resource  Description  Framework 

RDECOM 

US  Army  Research,  Development  and  Engineering  Command 

RT 

Research  Task 

RTI 

Runtime  Infrastructure 

RFP 

Request  for  Proposal 

RPM 

Revolutions  Per  Minute 

RPR  FOM 

Real-time  Platform  Reference  Federation  Object  Model 

ROI 

Return  On  Investment 

SAVI 

System  Architecture  Virtual  Integration 

SE 

System  Engineering 

SERC 

Systems  Engineering  Research  Center 

SETR 

System  Engineering  Technical  Review 

Simulink/Stateflow 

Product  family  for  model-based  control  system  produced  by  The  Mathworks 

SCR 

Software  Cost  Reduction 

SDD 

Software  Design  Document 
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SE 

System  Engineering 

SFR 

System  Functional  Review 

SISO 

Simulation  Interoperability  Standards  Organization 

SLOC 

Software  Lines  of  Code 

SME 

Subject  Matter  Expert 

SOAP 

A  protocol  for  exchanging  XML-based  messages  -  originally  stood  for  Simple 
Object  Access  Protocol 

SoS 

System  of  Systems 

Software  Factory 

Term  used  by  Microsoft 

SPARQL 

SPARQL  Protocol  and  RDF  Query  Language 

SRR 

System  Requirements  Review 

SRS 

Software  Requirement  Specification 

SST 

Single  Source  of  Truth 

SSTT 

Single  Source  of  Technical  Truth 

STOVL 

Short  takeoff  and  vertical  landing 

SVR 

System  Verification  Review 

SW 

Software 

SWT 

Semantic  Web  Technology 

SysML 

System  Modeling  Language 

TARDEC 

US  Army  Tank  Automotive  Research 

TBD 

To  Be  Determined 

TRL 

Technology  Readiness  Level 

TRR 

Test  Readiness  Review 

Turtle 

Terse  RDF  Triple  Language 

UAV 

Unmanned  Aerial  Vehicle 

UAS 

Unmanned  Aerial  System 

UC 

Use  Case 

UCAV 

Unmanned  Combat  Air  Vehicles 

UML 

Unified  Modeling  Language 

Unix 

An  operating  system  with  trademark  held  by  the  Open  Group 

UQ 

Uncertainty  Quantification 

US 

United  States 

USD 

US  Dollars 

use 

University  of  Southern  California 

VHDL 

Verilog  Flardware  Description  Language 

VR 

Virtual  Reality 

V&V 

Verification  and  Validation 

XMI 

XML  Metadata  Interchange 

XML 

extensible  Markup  Language 

XSLT 

extensible  Stylesheet  Language  family  (XSL)  Transformation 

xUML 

Executable  UML 
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18  Trademarks 


Analysis  Server  is  a  registered  trademark  of  Phoenix  Integration,  Inc. 

Astah  SysML  is  Copyright  of  Change  Vision,  Inc. 

BridgePoint  is  a  registered  trademark  of  Mentor  Graphics. 

Cameo  Simulation  Toolkit  is  a  registered  trademark  of  No  Magic,  Inc. 

CORE  is  a  registered  trademark  of  Vitech  Corporation. 

CREO  is  a  registered  trademark  of  PTC  Corporation. 

DOORS  is  a  registered  trademark  of  IBM  Corporation. 

IBM™  is  a  trademark  of  the  IBM  Corporation 
iGrafx  is  a  registered  trademark  of  iGrafx,  LCC. 

Java™  and  J2EE™  are  trademark  of  SUN  Microsystems 
Java  is  trademarked  by  Sun  Microsystems,  Inc. 

LDRA  is  a  registered  trademark  of  Trademark  of  LDRA  Ltd.  and  Subsidiaries. 

Linux  is  a  registered  trademark  of  Linux  Mark  Institute. 

Mathworks,  Simulink,  and  Stateflow  are  registered  trademarks  of  The  Mathworks,  Inc. 
MagicDraw  is  a  trademark  of  No  Magic,  Inc. 

MATRIXx  is  a  registered  trademark  of  National  Instruments. 

Microsoft®,  Windows®,  Windows  NT®,  Windows  Server®  and  Windows  VistaTM  are  either 
registered  trademarks  or  trademarks  of  Microsoft  Corporation  in  the  United  States  and/or 
other  countries.  ModelCenter,  is  a  registered  trademark  of  Phoenix  Integration,  Inc. 

Modelica®  is  a  registered  trademark  of  the  Modelica  Association. 

Object  Management  Group  (OMG):  OMG's  Registered  Trademarks  include:  MDA®,  Model 
Driven  Architecture®,  UML®,  CORBA®,  CORBA  Academy®,  XMI® 

OMG's  Trademarks  include,  CWM™,  Model  Based  Application  Development™,  MDD™,  Model 
Based  Development™,  Model  Based  Management™,  Model  Based  Programming™,  Model 
Driven  Application  Development™,  Model  Driven  Development™ 

Model  Driven  Programming™,  Model  Driven  Systems™,  OMG  Interface  Definition  Language 
(IDL)™,  Unified  Modeling  Language™,  «UML»™ 

OMG®,  MDA®,  UML®,  MOF®,  XMI®,  SysML™,  BPML™  are  registered  trademarks  or  trademarks 
of  the  Object  Management  Group. 

Oracle  and  Java  are  registered  trademarks  of  Oracle,  Inc.  and/or  its  affiliates. 

ParaMagic  is  a  registered  trademark  of  InterCAX,  Inc. 

PHX  ModelCenter  is  a  registered  trademark  of  Phoenix  Integration,  Inc. 

PowerPoint  is  a  registered  trademark  of  Microsoft,  Inc. 

PTD  is  a  registered  trademark  of  PTC  Corporation,  Inc. 

Real-time  Studio  Professional  is  a  registered  trademark  of  ARTiSAN  Software  Tools,  Inc. 
Rhapsody  is  a  registered  trademark  of  Telelogic/IBM. 

Rose  XDE  is  a  registered  trademark  of  IBM. 

SCADE  is  copyrighted  to  Esterel  Technologies. 

Simulink  is  a  registered  trademark  of  The  MathWorks. 
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Solidworks  is  and  3DEXPERIENCE,  the  Compass  icon,  the  3DS  logo,  CATIA,  SOLIDWORKS, 
ENOVIA,  DELMIA,  SIMULIA,  GEOVIA,  EXALEAD,  3D  VIA,  3DSWYM,  BIOVIA,  NETVIBES,  and 
3DEXCITE  are  trademarks  or  registered  trademarks  of  Dassault  Systemes. 

Stateflow  is  a  registered  trademark  of  The  MathWorks. 

Statemate  is  a  registered  trademark  of  Telelogic/IBM. 

STK  is  a  registered  trademark  of  Analytical  Graphics,  Incorporated  (AGI),  Inc. 

Syndeia  is  a  product  of  Intercax  Corporation. 

UNIX  is  a  registered  trademark  of  The  Open  Group. 

VAPS  is  registered  at  eNGENUITY  Technologies. 

VectorCAST  is  a  registered  trademark  of  Vector  Software,  Inc. 

Visio  is  a  registered  trademark  of  Microsoft,  Inc. 

VT  MAK  is  a  product  of  VT  Systems,  Inc. 

VxWorks  is  a  registered  trademark  of  Wind  River  Systems,  Inc. 

Windchill  is  a  registered  trademark  of  PTC,  Inc. 

Windows  is  a  registered  trademark  of  Microsoft  Corporation  in  the  United  States  and  other 
countries. 

XML™  is  a  trademark  of  W3C 

All  other  trademarks  belong  to  their  respective  organizations. 
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Change  Log 

This  is  the  fifth  version  of  this  report  that  used  an  evolving  interim  report  of  the  Systems 
Engineering  Research  Center  (SERC)  research  task  (RT)  RT-168.  We  used  an  evolving  interim 
report  as  a  means  to  communicate  the  bi-monthly  status  in  order  to  relate  and  integrate  the 
results  from  our  large  team  of  researchers  working  on  several  challenge  problems.  The  bi¬ 
monthly  status  information,  bi-weekly  presentations,  technical  reports  and  demonstrations  are 
listed  Section  2.  This  report  presents  additional  task  details  in  Part  II.  The  evolution  of  the 
report  an  specific  changes  are  included  in  a  change  log. 


Change  Log 


Date 

Deliverable 

Change  Items/Comments 

Nov  1,  2016 

Delivered  first 

Interim 

Technical 

Report 

Provided  for  bi-monthly  as  described  above  in  preface. 

Jan  17,  2017 

Delivered 

second  Interim 

Technical 

Report 

Feb  6,  2017 

Eddie  Bauer  requested  that  this  change  log  be  added  to  the 
document. 

Mar  15,  2017 

Delivered  third 

Interim 

Technical 

Report 

Updates  listed  below  map  primarily  to  research  use  cases  (UCxx) 
shown  in  Figure  3.  Details  associated  with  these  items  are  included  in 
the  respective  sections  for  each  use  case. 

Scope  updated  to  include  new  Challenge  Area  #5,  and  briefing 
summaries  of  inputs  from  ARDEC's  IPT  briefings. 

Section  2.3  summarizes  working  sessions  and  sponsor-supporting 
events,  updates  to  actual  events,  presentations,  and  demonstrations, 
and  plans  for  upcoming  events. 

UC01  graphical  CONOPS  demonstrator  of  autonomous  UAS  demo 
now  exposing  parametrics  for  MDAO  at  this  level. 

UC01  demonstration  and  data  structure  analysis  for  Early  Synthetic 
Prototyping 

UC01-UC06  Framework,  Syndeia  demonstration. 

Paul  Grogan  integrating  Semantic  Web  Technology  with  modeling  and 
simulation  case  study  for  UC02  this  should  also  fit  into  the  new 
framework  concept  discussed  in  Section  12.8. 

UC03  is  expanding  MDAO  examples  to  include  mission  and  subsystem 
as  well  as  system  analysis. 

Updates  to  UC06  Decision  Framework  and  AAMODAT,  is  now  related 
to  a  new  challenge  area  #5  that  is  being  refined  under  UC10  in  Section 
13. 

Expanded  UC09  and  changed  name  to  Tradeoff  Analysis  of 
Technologies  for  Integration  or  Interoperability  to  discuss  approach  to 
analyzing  tool  integration,  relationships  to  Canonical  Reference 
architecture  of  an  Integrated  MCE  Environment,  Windchill  Analysis, 
Syndeia,  OpenMBEE  and  Open  Collaboration  Group  for  MBSE,  Digital 
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Environment  at  Airbus  Space,  RT-168  Tool-to-Tool  Integrating  and 
Interoperability  Framework. 

Added  use  case  UC10  to  explicitly  describe  research  efforts  in  using 

semantic  web  technologies  to  create  inputs  to  AAMODAT _ 

Added  section  on  research  synergies  as  this  was  one  of  the  early 
requests  identified  by  the  research  sponsor.  For  example,  as  a  results 
NAVAIR  and  ARDEC  are  discussing  needed  agreements  such  as  a 
memorandum  of  understanding  to  support  sharing  of  research  results 
and  technologies.  Collaboration  with  OpenMBEE  community,  and 
Aerospace  Industry  Association  on  CONOPS  for  Government  and 
Industry  collaboration  through  MBSE 

July  10,  2017  Delivered  This  update  blends  information  from  the  3rd  and  4th  working  sessions, 

fourth  Interim  the  March-April  bi-monthly  status  reports  and  the  bi-weekly 

Technical  meetings,  presentations  and  demonstrations. 

Report  General  updates  to  Executive  Summary  and  Introduction  sections, 

including  identifying  new  collaboration  and  working  group  efforts. 
Updates  to  Section  2.4  provide  summary  of  demonstrations, 
presentations,  reports,  deliverables  and  meetings  from  March 
through  July  10. 

Minor  updates  to  Section  2.1  to  reflect  publically  released  findings  for 
Challenge  Areas  #1 

Updated  the  use  case  research  figure  (Figure  3)  to  show  new  efforts 
and  stakeholders;  this  figure  also  updates  the  associated  description 
of  those  changes  in  Section  2.2  where  the  use  cases  are  summarized, 
with  a  brief  bullet  or  two  about  the  latest  updates  and  newest 
research  investigations.  The  use  case  details  are  provided  in  Part  II  of 
the  report  (Sections  3  though  14). 

Added  UC11  as  "Assess  AVCE  iMBE"  with  new  team  member  Rick 
Dove  to  perform  this  analysis  in  the  context  of  INCOSE's  Agile  Systems 
Engineering  Life  Cycle  Model  (ASELCM)  project,  and  specifically  in 
terms  characterized  by  the  ASELCM  Pattern  of  Three  Concurrent 

Systems. _ 

Section  2.3  -  added  summary  of  working  sessions  3  and  4 _ 

Section  2.4  -  updated  summary  of  events,  demonstration  and 

deliverables  by  date _ 

Section  3.1  provides  more  information  about  UC00  -  Information 
Model  and  its  relationship  to  Semantic  Web  Technologies  (SWT),  the 
NASA/JPL  ontologies/SWT,  and  the  way  that  SWT  capabilities  are 
embedded  within  a  NASA/JPL  instantiation  of  OpenMBEE.  This 
information  has  been  presented  in  several  different  working  sessions, 
and  we  have  demonstrated  the  concept  as  part  of  the  Integration  and 

Interoperability  (lolF)  framework  (UC09). _ 

Section  4.1  has  updated  details  on  the  Graphical  CONOPS,  which  were 
summarized  at  4th  working  session;  this  includes  updates  to  the 
functionality,  interface,  controls,  and  parametric  details  that  are  going 
to  be  analyzed  using  MDAO. _ 

_ Section  4.2  discusses  information  provided  by  USC  collaborator  about 
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the  metadata  extracted  from  the  Early  Synthetic  Prototyping  work  to 
inform  the  information  model  (UCOO). 

Section  5.2  discusses  the  initial  demonstrations  for  integrating  the 
outputs  from  the  graphical  CONOPS  simulation  into  the  2D 
simulations  using  the  lolF.  A  second  concept  added  to  the  lolF 
demonstrated  a  simplified  example  to  show  the  data  exchange 
through  SWT  component  of  lolF.  This  updated  information  could  have 

been  placed  with  either  UCOO,  UC01  or  UC09. _ 

Sections  6.4.2  and  6.5  discuss  new  examples  of  MDAO,  including 
more  workflows,  in  addition  to  the  use  of  MBSE  Analyzer  to  integrate 
SysML  models  with  ModelCenter. 

Section  6.6  discusses  new  research  for  applying  MDAO  to  graphical 
CONOPS  and  MDAO  to  support  formalizing  Assessment  Flow  Diagram 
for  the  Decision  Framework  and  AAMODAT.  See  more  details  in 

Section  9.2. _ 

Section  7.1  describes  the  research  into  the  application  of  OpenMBEE 
and  specifically  the  Model  Development  Kit  (MDK)  DocGen 
component,  where  we  have  developed  a  number  of  View  and 
Viewpoint  hierarchies  for  using  DocGen,  including  generation  of  the 

"specification"  for  AVCE  iMBE. _ 

Section  7.2  is  a  new  section  that  discusses  basic  concept  of 
MDK/DocGen. 

Section  8  summarized  various  presentation  provided  in  the  context  of 
MBE  and  manufacturability  made  by  Kishore  at  working  sessions  and 
bi-weekly  meetings. 

Section  8.4  is  new  and  reflects  on  two  talks  that  were  provided 
related  to  Automated  Concurrent  Engineering  (ACE)  and  a  new 
Section  8.5  that  discusses  extension  to  the  ACE  concept  using  SWT 
that  we  are  trying  to  align  with  lolF. 

Added  Section  8.6  to  describe  the  MBE  analysis  performed  to  support 
more  realistic  battery  performance  for  the  graphical  CONOPS 
quadcopters  (UC01). 

Added  a  new  Section  9.2  to  discuss  the  SWT  application,  and 
relationships  to  the  Decision  Framework  and  AAMODAT.  This  section 
also  provides  minor  updates,  including  relating  back  to  Section  6.7, 
which  discusses  the  concept  for  using  MDAO  workflows  to  represent 
the  Assessment  Flow  Diagrams  in  a  formal  way  that  could  be 
automatically  extracted  from  SysML  models  to  populate  SWT  for 
automating  the  population  of  AAMODAT. 

Minor  updates  to  some  of  Section  11  on  some  opportunistic 
relationships  with  other  research  to  provide  support  for  V&V. 
Concepts  related  to  these  V&V  efforts  are  planned  for  a  SERC  event  in 
early  December  2017. 

Minor  updates  to  Section  12,  but  much  of  the  advancements  relate  to 
the  evolving  lolF,  and  many  of  the  related  updates  are  now 
interspersed  throughout  the  other  parts  of  the  document,  because 
the  lolF  is  now  beginning  to  bring  many  of  the  research  elements 
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together. _ 

Section  13  updates  cover  the  demonstrations  and  discussions  at  the 
third  working  session  on  using  SWT  as  a  means  for  collecting  and 

providing  automated  inputs  to  AAMODAT. _ 

Added  Section  14  for  the  new  use  case  UC11  on  Assessing  AVCE  iMBE 
based  on  the  concept  of  Agile  Systems  Engineering  Life  Cycle  Model 
(ASELCM)  project.  Specifically,  the  ASELCM  Pattern  of  Three 

Concurrent  Systems. _ 

Updates  to  Section  15.5  on  Semantic  Technologies  for  Systems 
Engineering  Working  Group. 

August  8,  Delivered  Final  General  updates  throughout  the  document  to  format  like  a  Final 

2017  Technical  Technical  report  vs.  an  evolving  interim  report.  This  includes  moving 

Report  this  change  log  to  a  final  section  that  will  not  be  included  in  the  final 

pdf  version  of  this  report  that  will  be  posted  on  the  SERC  website. 
Added  accomplishment  summary  to  bottom  of  Executive  Summary 

Updated  part  II  number. _ 

Update  to  the  deliverables  and  event,  and  demonstration  summary, 

and  as  appropriate  discuss  in  the  use  cases  those  new  advances. _ 

Had  minor  updates  to  almost  all  sections  where  new  demonstrations 
were  conducted  from  working  session  #4  to  working  session  #5. 

Updates  from  USC/ICT  inputs  added  to  Section  4.2. _ 

Added  summary  of  information  presented  in  the  demonstration  at 
working  session  #5  on  the  Section  5.2 

Update  Section  15  to  discuss  evolving  synergies  including  the  special 
session  on  July  31st  that  brought  in  several  other  service  and  DoD 
organizations. 

Update  summary  with  list  of  accomplishments. _ 

_ Updated  acronyms  and  abbreviations. 
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